![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
tugrikThis one is for the programmer and systems types, as it's a bit of a tech-nerdy rant.
So there I was, pulled into a project that's months along and 80% done, to help them solve a few problems that were keeping the last 20% from finishing. The project is an HP Openview based monitoring system that we're going to use to monitor our network -- and we're also going to give a subset of it to our customers. The first problem they present sticks up like a loose thread. A quick exploratory tug suddenly makes a huge section of the project's fabric unravel.
The next one is even worse.
Within 30 minutes of very heated chatter we've pretty much invalidated 3 months of work from very expensive contractors and a handful of in-house engineers. What happened is best described as utter inelegance caused by engineers seeing things from a coding standpoint only and utterly ignoring how the project looks from the user and customer standpoint. They created an almost unusable interrface: it gives an incredibly complete view of every single parameter in our network (in a way the most anal of anal retentives would love) while managing to obfusciate what a user of the project would need to know. Compare it to a full description of how to build an engine, down to the torque of every single screw, while failing to tell the user (driver) of the car "insert key and turn" to start the car.
When I called them on it every single one of them got their hackles up as if I was attacking their children. Most of them attributed my concerns to "I just don't understand". As I wasn't their particular brand of engineer I couldn't possibly see the Truth(tm), and so I, just like the users to come after me, would have to be browbeaten into submission instead of daring to ask them to rewrite things to where it was actually usable. Unfortuantely for them I do know the intricate ins and outs of what they were doing and I know how the users need to access the data, so each time an attempted brow-beating began I was able to pluck it smooth and throw it right back at them.
It got ugly, yeah.
Still, I'm backing off. My complaints are noted, and in fact I've started rebuilding their project from scratch from a user perspective, to show the bossmen later when the one they're doing bombs in their face. It's just sad to see so much time and money wasted to do little more than preserve the ego of folks who refuse to see things from a "lowly" customer or end-users's POV. Ahwell. It just leaves me frustrated. It also leaves me with the urge to just dive in and do it all myself, which given my workload with the rest of the network is utterly impossible. Sure, I can make a demo system to show how it SHOULD be done (as I just mentioned), but not a full implementation.
At the same time, I'm desperately wishing someone would prove me wrong. If I'm the one being short-sighted and can't see the forest for the trees, then I hope someone can coherently point this out to me so I can learn from it and help solve this turkey of a problem. My ego will gladly take the knocking-about and in fact would benefit from it. When you design complex systems that can get bigger than you can handle alone, being questioned and proven wrong is a wonderful thing.
As an example for the tech-head types, here's one of the bass-ackwards things in their design:
One item being monitored is a reference station. To operate, a reference station (depending on its location and duties) will have a subset of 6 services running. Each service performs a particular task and should always be running. When one service goes down you have a set amount of time (depending on the service) before Everything Is Bad. So it stands to reason that the monitoring system should check that each service is running and when they fail inform you which service went down.
Instead, they simply have a variable saying "this machine runs X services" (say, 5 of them). If one or more dies, their monitor goes "Hey! you're running less than (X) services!". It doesn't tell you which one died. It doesn't record any historical data of how long each one was down. It simply says that not enough were up.
It 'costs' more, network and coding wise, to do it their way. It presents a world less data. It's useless for the poor tech on the other end of the pager who gets "uh, not enough services running". They have to go inspect the server, check each and every service, and restart/repair any that failed. If they'd done it the right way the tech would get a "Service so-and-so is down". They'd know exactly where to look, exactly what to restart... and in fact, the monitoring system could even attempt to restart things itself, providing basic self-repair automation.
I suggested this, and it caused a screaming match between four engineers that strayed from english to russian to korean and back, each time getting more personal in the attacks. They calmed down only after they'd successfully quashed any chance of productive conversation... and the next item in the meeting was moved on to as if nothing happened.
I liken the whole thing to a road trip gone wrong. It's like asking those engineers how best to get from San Fran to LA in your car. One of them determined you might get better gas milage by driving in reverse... so they suggest going backwards down the highway while sitting on the hood of the car, windshield smashed through, steering with your feet and accelerating/braking with a jury-rigged contraption of popsicle sticks and rubberbands bought at a truckstop.
When you suggest "why not just put the car in "drive" and head down the highway" you get attacked -- because you can't possibly understand how to efficiently use your car like those engineers you asked can.
Hrmf.
So there I was, pulled into a project that's months along and 80% done, to help them solve a few problems that were keeping the last 20% from finishing. The project is an HP Openview based monitoring system that we're going to use to monitor our network -- and we're also going to give a subset of it to our customers. The first problem they present sticks up like a loose thread. A quick exploratory tug suddenly makes a huge section of the project's fabric unravel.
The next one is even worse.
Within 30 minutes of very heated chatter we've pretty much invalidated 3 months of work from very expensive contractors and a handful of in-house engineers. What happened is best described as utter inelegance caused by engineers seeing things from a coding standpoint only and utterly ignoring how the project looks from the user and customer standpoint. They created an almost unusable interrface: it gives an incredibly complete view of every single parameter in our network (in a way the most anal of anal retentives would love) while managing to obfusciate what a user of the project would need to know. Compare it to a full description of how to build an engine, down to the torque of every single screw, while failing to tell the user (driver) of the car "insert key and turn" to start the car.
When I called them on it every single one of them got their hackles up as if I was attacking their children. Most of them attributed my concerns to "I just don't understand". As I wasn't their particular brand of engineer I couldn't possibly see the Truth(tm), and so I, just like the users to come after me, would have to be browbeaten into submission instead of daring to ask them to rewrite things to where it was actually usable. Unfortuantely for them I do know the intricate ins and outs of what they were doing and I know how the users need to access the data, so each time an attempted brow-beating began I was able to pluck it smooth and throw it right back at them.
It got ugly, yeah.
Still, I'm backing off. My complaints are noted, and in fact I've started rebuilding their project from scratch from a user perspective, to show the bossmen later when the one they're doing bombs in their face. It's just sad to see so much time and money wasted to do little more than preserve the ego of folks who refuse to see things from a "lowly" customer or end-users's POV. Ahwell. It just leaves me frustrated. It also leaves me with the urge to just dive in and do it all myself, which given my workload with the rest of the network is utterly impossible. Sure, I can make a demo system to show how it SHOULD be done (as I just mentioned), but not a full implementation.
At the same time, I'm desperately wishing someone would prove me wrong. If I'm the one being short-sighted and can't see the forest for the trees, then I hope someone can coherently point this out to me so I can learn from it and help solve this turkey of a problem. My ego will gladly take the knocking-about and in fact would benefit from it. When you design complex systems that can get bigger than you can handle alone, being questioned and proven wrong is a wonderful thing.
As an example for the tech-head types, here's one of the bass-ackwards things in their design:
One item being monitored is a reference station. To operate, a reference station (depending on its location and duties) will have a subset of 6 services running. Each service performs a particular task and should always be running. When one service goes down you have a set amount of time (depending on the service) before Everything Is Bad. So it stands to reason that the monitoring system should check that each service is running and when they fail inform you which service went down.
Instead, they simply have a variable saying "this machine runs X services" (say, 5 of them). If one or more dies, their monitor goes "Hey! you're running less than (X) services!". It doesn't tell you which one died. It doesn't record any historical data of how long each one was down. It simply says that not enough were up.
It 'costs' more, network and coding wise, to do it their way. It presents a world less data. It's useless for the poor tech on the other end of the pager who gets "uh, not enough services running". They have to go inspect the server, check each and every service, and restart/repair any that failed. If they'd done it the right way the tech would get a "Service so-and-so is down". They'd know exactly where to look, exactly what to restart... and in fact, the monitoring system could even attempt to restart things itself, providing basic self-repair automation.
I suggested this, and it caused a screaming match between four engineers that strayed from english to russian to korean and back, each time getting more personal in the attacks. They calmed down only after they'd successfully quashed any chance of productive conversation... and the next item in the meeting was moved on to as if nothing happened.
I liken the whole thing to a road trip gone wrong. It's like asking those engineers how best to get from San Fran to LA in your car. One of them determined you might get better gas milage by driving in reverse... so they suggest going backwards down the highway while sitting on the hood of the car, windshield smashed through, steering with your feet and accelerating/braking with a jury-rigged contraption of popsicle sticks and rubberbands bought at a truckstop.
When you suggest "why not just put the car in "drive" and head down the highway" you get attacked -- because you can't possibly understand how to efficiently use your car like those engineers you asked can.
Hrmf.
![[identity profile]](https://www.dreamwidth.org/img/silk/identity/openid.png){kind=small}
no subject
Date: 2003-10-21 06:43 pm (UTC)In fact when I suggested that he just go for engineering instead of construction management, he gave me the look, the look of "why would I want to go backwards instead of forwards?" Like I was a small child asking why I couldn't have cookies before dinner.
(Yet my dad, an electical engineer, is a reasonable person. He, however, did not go to college. Maybe engineering college is what does them in?)
I know. Odd tangents. It's gonna be one of those months. ;)
no subject
Date: 2003-10-21 06:57 pm (UTC)Probably the best thing to do is to run a drill... Have them set up the system, then discreetly pull the plug on something somewhere and see how long it takes them to notice something's wrong.
On the other hand, if it *works*, well, then clearly they failed to explain something, but now you'll be able to document what that is.
Interface requirements spec.
Date: 2003-10-21 07:45 pm (UTC)It will then become bleedingly obvious to them why the system doesn't work. As Tuftears said, it sounds like they have no clue how a system like this would actually be used.
Speaking as an "engineer-in-training" (that's what we're supposed to call ourselves until we get the other piece of paper), engineers will obsess over how to elegantly solve problems. Problems arise when they're not solving the problem you want them to solve. Change the target problem, and they'll actually start to _help_ you. Hopefully }:>.
-Deuce
no subject
Date: 2003-10-21 07:56 pm (UTC)I'm doing a project. My first task was to work out what needed to be done. My second task was to create, and get approved, a user interface that my users can understand.
Only after that did I start writing code.
no subject
Date: 2003-10-21 08:29 pm (UTC)Somebody wanted him to machine a J-bolt, and its final destination was a block of (already cured) concrete. The other student took half an hour to clue in to "How would it get in there?" before retreating, shamefaced. There was much snickering after the student left for the day. It sounds like your engineers can't understand the J-bolt and the concrete block.
Been there, seen that
Date: 2003-10-21 08:35 pm (UTC)Most of the VoIP protocol stuff I've seen is very much done by the programmers for the programmers. Management, billing, security, usability... none of it is considered until afterwards in the design. (*)
I suppose in some ways it is a reflection the feeling among some that programming is still an art more than an engineering discipline, with folks not understanding this is supposed to be, say, commercial art which is targeted to be understood by a specific group, not 'fine' art for its own sake. ("I don't get how this shapeless blob of color is art"/"I don't understand how to use this interface." ==> "That's because you're an unsophisticated non-artiste/non-engineer.")
Not that striving for engineering elegance and programming elegance for itself own sake is bad per se, just this isn't what most software engineers are being paid to do -- the task being paid for is to satisfy the customer/user/requirements first, and yourself, hopefully. A beautiful, elegant, pleasing solution that does not meet the requirements is not a solution. :)
(* -- Frequently heard during code/design reviews of my VoIP protocol parser: "That's huge! I could write something in a tenth of the size that did all that using this other method/style." Me: "Yah, but what if someone did this in a message, then this, then that, and finally that." "... Well, I'd never send a message like that. Who would?" In one ear, out the other...)
no subject
Date: 2003-10-21 09:09 pm (UTC)Yeah, it'd be nice to say it...
no subject
Date: 2003-10-21 09:26 pm (UTC)Pertinent Reading
Date: 2003-10-21 09:36 pm (UTC)Think I've got a spare around, if you might be interested.
no subject
Date: 2003-10-21 09:42 pm (UTC)I remember once heading over to the Q.A. lab and a couple engineers were chatting with the techs there. During conversation, a couple of the odd quirks customers dealt with were brought up. One of the engineers flat out stated, "Our boxes don't do that." When I countered with indeed, they do, he refused to believe it. So I walked over to one of the test machines and duplicated the problem. "It's not supposed to do that." Well duh! Maybe if they listened more often. :)
Of course, this was seperate from the pissing matches engineers got into with support, saying we had no idea what we were talking about after we loaded them with a bunch of evidence showing they were wrong. Sometimes they's so pig headed!
(P.S. Might be a good post for a userpic? ;) )
Right-side programming...
Date: 2003-10-22 01:09 am (UTC)And then you get your work critiqued by someone who's never touched your code....
...I believe that's my theory on why so many programmers are like that.
And I love the icon. Very Tug.
-Traveller
no subject
Date: 2003-10-22 01:51 am (UTC)Re: Pertinent Reading
Date: 2003-10-22 06:12 am (UTC)Fortunately, or unfortunately, I end up encountering my end users, so all of my control and monitoring systems have to make sense to them, and if they don't like how it works or looks, I'm stuck there changing it on the fly till they do.