SharePoint is the new MS Access

August 10, 2010

You know there is a type of program that every programmer dreads. That is the one that isn’t a program at all; it is a spreadsheet heaving with formulae or worse an MS Access database. The worst of the worst of these beasts is saving data straight into the database and just bypassing all your nice source-control system and all your best efforts at version control (and you can’t search it because it is all embedded in a binary file).

This stuff made by Lay Programmers is the foundation of many a system, but at some point it goes wrong and Real Programmers have to “just tidy up a bit” and put in some foundations and some structural steel. But, it takes years to get rid of them; especially if there are other 2-tier beasts that just go into the database. It probably takes a year to capture most of the use-cases into a “proper” client UI or a service and there are always some that need to be entered manually. Although the corner-cases only emerge at the end of a year;and that means calendar year-end, financial year-end and Australian year-end so you might as well wait another year.

Of course, we have all got rid of Access and don’t let people do that kind of thing. Hooray!

Except that we now have the SharePoint to deal with. Easy to set up as long as you don’t really want a backup and don’t mind that all the config and customisations are trapped on one machine. So easy to set up another site that you don’t bother to do a good job as you can just set another site up with another set of lists that is 85% the same as the last one. And we can finally pursue the dream of non-programmers making their own workflows. Yippee. Except when they need to be extracted and turned into the real thing. So easy to get so much mission critical data stored there that is really crucial and really hard to export.

So much about SharePoint is nice, the fact that 75% of data storage CRUD is done out-of-the-box and all in a web interface that is simple to use and integrated with Excel. It’s that last 25% that will keep me in programming work. I bet that all of these things are possible; of they will be if I pay a SharePoint consultant a fortune to fix it and if I just upgrade to the new version all this will be much simpler…

Developer shibboleths and secret handshakes

July 28, 2010

Some parts of interviewing are positive. Letting people recount their experiences and skills that they already possess; letting them tell you about their hopes for the future. Then, there is the negative side; trying to find the fraudsters and the pretenders. These pretenders can have legitimate reasons for not knowing stuff; they might be a raw graduate; they might be smart but only getting started in programming; they might have good, transferable experience from another technology but you just aren’t asking the right questions; they might be nervous or undermotivated or a host of other problems.

Sometimes finding these people comes down, for me, for just a few words. They say something and it just sets off alarm bells: a programmer would never say that. Not having the right skills is fine; not knowing the right word is fine as long as they have the right model in their heads. Sometimes it is even a single word used incorrectly; a shibboleth. My wife, who is a clever woman and an able user of technology but really, really not a technical person, often uses the word “database” to mean “a store of information” whether that is a list of things in a spreadsheet, a collection of files in a folder, a collection of documents in a content management system or anything else. All true, but a developer would never say that without some serious explanation about how it wasn’t a real database. The one that really gets me irritated is referring to “codes” rather than code.

Of course, the risk is that I’m recruiting someone just like me. That might work for some things but I suspect tends to build a rather dangerous monoculture of People Like Us. Some common frame of reference is necessary to build respect; assuming the other person is good until proved otherwise is fine for teenagers, but I’m too old to waste time on that. And yet, only trusting people that are like you is one step up from a secret handshake. Attach the rock of triumph!

Applications that survive are bad or good?

July 10, 2010

I recently was discussing the first complete application that I wrote professionally. It does a whole mess of stuff; data retrieval from several systems, does some calculations, makes some reports*, sticks some derived data in another place. You know the usual stuff.

So, it has been a number of years since I wrote it and it is still in daily use; so, I was inclined to crow a little bit and boast about its longevity and how it is still working well without any modification. Then, as pride comes before a fall, I realised that basically the only reason that it was still in use without any modification is that everyone was too scared to change it because it was such a ghastly, tottering heap of poop. OK, from hero to zero in an hour. Great.

Of course, it isn’t all bad. It has at least been worth keeping rather than re-writing, and I’ve seen a lot worse since. It still gets the answers right; but, IMHO, correct output is the weakest measure of software “goodness”. If it is “right” but not maintainable it is only a matter of time before the world moves on and your app is dead, dead, dead.

* funnily enough, the reporting part actually got turned into a mini “framework” for automating Excel**  (I know, I know, I’m blushing). And even now, I still see reports on people’s screens that have used this thing and so my visual tastes and simple table dumping aesthetics are hanging around meme-like on many a app.

** Really funnily though, when I started the app I didn’t use Excel, I did the super-quick “dump to CSV and I’ll sort it later” thing. By some evil, this got into the application stack in places it shouldn’t and things were *blush* being passed around as comma-separated strings.

Storing application config in the network

July 10, 2010

In his post on using the network as a place to store config configuration, Udi Dahan suggests that we use separated subnets when we want to have test setups with the same configuration as the production setup. I’ve seen some similar stuff to what Udi suggests.

1) Using cnames that have meaning pushes semantics into the network (i.e., DNS server)

Of course the real server name is something like uk_site002_prod_win98_2 and that has meaning to someone in the netadmins (that is, unless they have settled on one naming convention per machine) but we really like it to have meaning in the context of our application stack. So we get a name like CRM_APPSERVER which is readable and can be used across the organisation. And we stand a chance of using it to swap out to other machines when we really need to, like during a crisis.

2) Using local cname resolution for testing on developer machines

The use of cnames has another thing. When you are testing things locally you can go to your HOSTS file to map the meaningful cname to another machine. In my case that is often localhost. This works really well for me when we are using SQL server linked database servers.

3) Using subnet resolution of cnames for multi-machine test environments (what Udi says)

So. You can take this to the next level and set up a small test subnet that has the cnames adjusted from a local DNS server that means you can keep the same names and they resolve to machines on the test network. Probably at this stage you have the netadmins make you really safe and ensure that you can’t even route the traffic out of the subnet.

4) Using Active directory as a place to look up *everything* not just semantic machine names

I’ve seen this taken one stage further to have Active Directory used as a supergiant, cross-process, cross-machine singleton. Basically, it was used to store the application config for the whole domain. Problems were it became rather cluttered with dozens of settings and the ease of access to the config meant that all levels of the application stack were able to just reach out and get it! That, of course, wasn’t the fault of the config. You could do quite cunning things coupling this with AD permissions, child domains and so on. Meaning you could do things like drop binaries onto the machines from the outside, but the machines inside can never get out even if they try.

Sacrificial wear in software

June 3, 2010

There is a concept in mechanical engineering called sacrificial wear. When two mechanical parts come into contact, they will inevitably eventually damage each other beyond use and have to be replaced. This is life for mechanical engineers.

However, part of the problem of replacing something is that you need to spend time taking it apart and fixing it; and worrying about where all the little bits of ground up metal have gone. Also, the problem is that both parts get worn; and might, in fact, wear out at slightly different rates so you might have twice as many failures. So if we design something in such a way that one of the two parts is made of softer metal you know it will wear faster. So, you can 1) know that it will always be the soft part that wears, the hard component will “never” wear; 2) you can optimise your design so that the soft part is easy and cheap to replace; all the clever stuff is in the bit that never wears. One example is Shimano SPD pedals. The clip on the pedal is clever and made of hard metal; the flange on the shoe is softer, simple to replace and very cheap, being just a square of steel with a hole in the middle.

This is one of the reasons why mechanical engineering is harder than software. After all you don’t need to worry about that for loop wearing out. Or do you? In fact, I think that there is some use in this kind of thinking. We don’t have code that rusts; but it does, when subjected to enough corroding changes, have faults that we think of as “structural”. We don’t have class that needs to have a big enough hole in it to get a spanner inside to change the grommets; but we do need to make extension points for our classes so that they can be extended.

Config files are the simple extreme of sacrificial wear; we “know” we can change config without damaging the application. The config file is part of the software and there should be no reason to treat it differently except that it has been designed to absorb a certain type of common change. IoC config files challenge the limits of the config file and behaviour that is “normally” contained imperative code creeps into a file that we call config.

Another type of sacrificial wear is the pattern I call “crap shield”; it’s somewhat like facade, but it’s purpose is more explicitly to hide any and everything that I don’t like. It can contain many facades; all the web references; all the links to dying databases; all the millions of project references that are needed to support a big 15 year old system. It keeps the code clean in my project though; when there is a problem with the “everything else” I know where to look.

Drag ‘n’ drop

May 1, 2010

An insight into GUI design: when discussing whether a particular screen needed drag ‘n’ drop, Stephen pointed out that drag ‘n’ drop only makes sense when there is more than one place you can drop the thing. In this case there were 2 lists that were being manually merged into 1. So 2 places to drag and 1 to drop. The more types of things to drag and the more places to drop the better; because the physical analogy is much better.

Of course, there is the other sort of drag ‘n’ drop; the type that is used for ordering items in a single list. For instance browser tabs. That lacks the physical analogy of dropping as you can’t drop it anywhere. I can’t think of a great name but drag ‘n’ snap comes close.

Project Management and the illusion of control

April 12, 2010

When you are working in an agile way and another project is working in a waterfall –  in the beginning at least — agile always looks bad. “They are so organised,” people say “we should have a project plan like that”. Telling people that it will never turn out like that just looks like sour grapes.

The problem is that those Gantt charts look like a prediction. They give the illusion that because you have drawn the charts, things will turn out that way. And you — or your boss — will be disappointed if that isn’t the case. That is, unfortunately, one reason why people like them; someone can write a “go live” date and then shout at people if it ain’t done.

We all want this level of control, just the agile project believes that it won’t look like that. Hankering after it isn’t bad. That level of control is closely related to the peak of our hierarchy of needs. We want freedom of choice, and choice requires control. It doesn’t feel good to be at the mercy of circumstance. And, looking at it from the other side, a programmer who says “It’ll be done when it is done” is really saying “Go away, you won’t understand and I don’t want you to. Programming is My Precious.”

What project management should be about is measurement. If you only ever have measurement then you are a least going to know what is currently going on. If you only have a “prediction” in the form of a Gantt chart that you made up at the start of the project but you don’t know what the current state is then that is much worse. The trick of predicting the future by yesterday’s weather is better than no prediction at all.

The question is, why can’t we estimate software? Personally speaking, I find that even the smallest unit of estimation is routinely off by 100%. Why is that?

Single Responsibility principle for checkins to source control

April 11, 2010

This is probably quite an obvious thought, but it occurred to me when trying to review a checkin that was done by someone who just can’t help himself when it comes to cleaning up code. It’s an obsession, so whatever he is doing he can’t help but leave the code a little tidier. Lovely. Except of course when his 10 lines of changes are obscured by the fact that he has deleted dead code from a hundred places in the file. Very hard to review.

So after that, we have a simple rule: if you must clean up, make a “null change” checkin for the cleanup with a comment to match; and another checkin with the fix. Then each checkin has a single responsibility. The mnemonic for the rule is: if you feel like using an “and” in your checkin comment you have probably done it wrong.

Metapost: blogging about blogging

February 13, 2010

I know it’s a sin; so I will be brief.

2 years, 63 posts, 2000+ page views on the most popular page.

It ain’t Joel on Software as most of the people reading this are sympathy readers :) but still…

Too detailed and not detailed enough

February 13, 2010

This is a phrase that has recently entered my lexicon.

It was coined in a moment when discussing a design document. The offending document claimed to be a design doc but covered such minutiae as the XML schemas for ALL the messages. The level of detail was totally overwhelming for a design document, you couldn’t see the wood for the trees. You couldn’t get a sense of the intent. However, the document, as it stood was not implementable. It wasn’t so detailed that the code would be trivial; it didn’t cover every use case; it didn’t describe how the code would be structured.

An architecture document can lock down the implementation with only a few details. If it says “the system shall..” and there is only one way to do that within the reality constraints of time, money and organisational know-how then there is the choice made. Conversely, there are details that are crucial to the design. For instance the architect should always be aware of the real-world boundaries between systems; they might be formed by an OO interface or by the fact that this system is run by you and that system is run by another company.

Of course, it isn’t only architects that commit this sin. The deadly user or business analyst can make the same mistake. You should recognise it when people start saying “you should store X in the database…” when they haven’t even read Databases for Dummies.

Once you start seeing things that are too detailed and not detailed enough they are everywhere.


Follow

Get every new post delivered to your Inbox.