Servusamanu.com

As mentioned, I work as a network administrator.  As part of my duties I also support 66 libraries within a certain five county area.  My department, Computer Operations, maintains a number of services for the libraries some of which are free, some of which are charged on a per case basis, and some of which are contracted support for a duration.

A number of issues have come up over the last year in relation to our support mechanism, many of which have posed problems with our overall infrastructure.  I’ve come to compile a short list of warnings that seem to apply well beyond the specific contracts in question.

The biggest issue is a lack of clarity and understanding of what the contract actually entails.  We periodically have issues regarding whether jobs are in contract or out of contract.  The problem is not necessary a lack of technical knowledge on either end, but an ambiguity within the contract itself.  Issues cannot be resolved with a close reading because the contract simply leaves the questions unanswered.  The contract never should have been written as it is.

Having a process for mediating conflicts and making changes is equally important.  Our client libraries will work with us for decades or longer.  We need a process of responding to complaints and removing the ambiguities in the contract. Problems will always become apparent if enough time is put to a task, but there must a response when they arise.

Long duration contracts require greater responsibility for the support party.  It’s just a statement of truth.  When we support a network for a year long contract we are obligated to respond to every issue regardless of what it might be.  Further, we become responsible if and when problems arise regardless of whether it was preventable or not.  Per item contracts decrease the number of things brought to our door and remove the responsibility from our shoulders.  The downside is the clients will never request preventive maintenance.  (Similar problem exists within healthcare.)

The last big thing is keeping and giving out documentation.  Having a full record of what went on for who and how much it cost is invaluable.  Our record-keeping has been splotchy, at best, and the result is that no party has the slightest idea whether a contract has been fulfilled.  The organization is so infinitely haphazard that clients have had to ask us whether the work was performed, because we never gave them and they never requested any receipt of work done.  Without any accountability on either side, neither the clients nor the providers can improve.

I imagine few organizations have these sorts of issues to the extent that we do.  Library systems have their own unique niche within the world, but nevertheless I’ve learned from our problems and…hopefully…someday we’ll get them all solved.

Rolling out a new piece of software, or a new policy, or a new and exciting piece of technology usually tends to wind up being something of a disaster.  Rarely are all the players working together to make for a smooth transition and the result is a debacle that usually falls in the lap of someone like me: the generic network admin.  As a front line for complaints and questions, the botched rollout leads to an endless source of long, tedious busy work, stressed phone calls, and general annoyance.

Recently I took the point on the rollout and I’m happy to say it  came together.  During the two to three weeks it took to get everything in place I jotted down a few notes, so the next time I do this I know ducks I need to line up before the product goes live.

1. Test the Product

In the past, half the issues that have arisen are due to unexpected behaviors, not on the part of the users, but from the software itself.  Not every single thing can be fixed, but making sure the people that will be supporting the new device are solid with its workings is vital.

2. Dodge the Creep

The first, biggest, and highest commandment with any rollout is minimizing the surface area for problems.  Roll out as little as possible while still retaining the core functionality.  Roll the product or software out to as few as possible, while making sure the key players are all included.  Don’t stretch too far, don’t turn a small task into a big one.

3. Prep the Field

Drop hints that changes are coming.  A prepared staff is better than an unprepared staff.  Even if the users in questions are pleased by the changes, they’ll appreciate the advanced knowledge.  This is especially true with the people in crux positions.  This obviously includes certain managers, but also the more charismatic employees and the individuals most likely to cause problems.

4. Get Everyone on Board

Sadly, this is never truly possible and it’s really just addendum to the note above, but it’s an important thing to attempt.  People on the top need to support the decision and people on the bottom need to be given the tools to accept it.  Receiving feedback prior to a final rollout, offering up training, and getting a few strong voices of supporting in the ranks before the rollout makes the whole process go a lot smoother.

5. Plan the Rollout

All the preparation in the world can dissolve into nothing if the last day becomes chaos.  The steps need to be outlined long before the final day.  An introductory email should be written and tested, a process for manually introducing the software should be in place, a response network to complaints and questions has to be ready for the first day.

6. Back it Up

The final thing and the most important for an extended project is holding the line.  There will almost always be complaints and they have to be dealt with, without compromising the original project.  A person prefers to use the old software-response: we’ll train you on the new software, but you still have to use it.  Allowing for loopholes and cracks turns into a support nightmare, undermines the new product, and makes a mess of the whole process.  A strong line has to be taken and held in order to have a successful rollout.

My rollout (of spiceworks and a few other things) have gone smoothly.  The best thing was dropping hints and giving small contained demonstrations long before I was ready to push the thing.  I showed a handful of people I knew would likely to either use the software or have some leverage over those who would or would simply be responsive.  In the end, when the big day came, the process was smooth and pretty damn well flawless.  Result: I get to sick back and let out a big sigh of relief.

I’m currently working as a network admin.

For the nerds out there:

8 FreeBSD Servers
1 Debian Server
1 Centos Server (1 more on the way)
2 Windows Server 2000 Boxes
1 Windows Server 2003 Box
1 Solaris Box

Email: PostFix
Monitoring: Nagios 3.0.6 among other things
Backups: RShapShot
WebServer: Apache 5

I’m working on moving things to Centos and virtualizing the network. I also want to get the email moved over to Zimbra. Users: 30 local users, 50 ‘offices’ with up to 10 users each. W00t w00t!