It wasn’t a planned vacation. On Tuesday evening, about 5:40 p.m., we lost
our staff’s e-mail connection to our Outlook Exchange server. Ten minutes later we lost the
attorneys’. We were victims of buzzard’s
luck: Two RAID discs and the controller in our Exchange server —the replacement
of which had been postponed to avoid a weekend disruption during preparation
for a trial — all gave up the ghost in rapid succession.
As our firm’s resident geek-attorney our IT Director, Sean,
sought my advice. Parts were on order,
but would not arrive until the following afternoon. The Internet, our network, and every other
system were purring like a kitten, however there was a high probability we going
to be without e-mail for at least most of Wednesday.
Wednesday being “Donut Day” at our firm, I suggested
hanging a sign on the door: ‘It’s e-mail-free Wednesday! Enjoy the donuts!’”
Sean, wisely, did not take this counsel.
“Okay, let’s pull out ‘Plan B,’” I suggested a little more
helpfully.
“Plan B” is our firm’s version of “Reverse 911.” Years earlier I had suggested that we harvest
our employee’s cell phone numbers so we could text-message everyone in case of
an emergency. The exigency we had in
mind at the time was a snow day, but to some in our office the absence of
e-mail would constitute an emergency and was certainly sufficient cause to
deploy Plan B.
"Plan B" |
I finally settled on this:
MW E-mail Services Are Down.
MW e-mail services crashed shortly after 5:40p this evening. IT is working heroically tonight to restore service, but restoration time is uncertain. You may need to rely on personal web-based e-mail on Wednesday. Incoming e-mail is being held at Postini. Take whatever measures you need to protect/alert of [sic] your clients.
Even this pithy prose had to be sent in three parts. I sent it twice because my Comcast webmail
kept falsely reporting that my efforts had been unsuccessful, notwithstanding
that they were appearing, almost instantly, on my iPhone.
That done, it being after-hours, and with no e-mail alerts
to intrude on my thoughts — only the soft glow of the yellow triangle with the
black exclamation mark emanating from my message bar declaring Outlook was dead
— I had time for a little personal techno-reverie:
I thought back to the earliest days of my practice. In 1981 written communications were
primarily made using an IBM Selectric II
typewriter and the U.S. Postal Service.
If something had to get somewhere in a hurry, you dashed to FedEx. Head, Moye, Carver & Ray owned a Qwip –an
early fax machine that was advertised
as being able to send one page every two
minutes (it was more like six) on a spinning cylinder, but only to another
Qwip machine. The Qwip —manufactured by
Exxon Office Systems, which also produced the cabinet-sized Vydec word
processor — was spectacularly unreliable, and was thus employed with the same frequency,
trepidation, and success as Get Smart’s
“Cone of Silence.”
Fast-forward about a decade.
When we first put IBM PS/2s on
attorneys’ desks, we couldn’t get them to turn them on. DOS
was an operating system only a geek could warm to, and few attorneys had the
keyboard skills, or patience, to learn WordPerfect. There was no e-mail because there was no commercial
Internet. When Banyan “Blue Mail” finally
arrived it worked only on our internal network, and no one read it.
My how things change.
Cobwebs grow on our in-boxes, which nowadays deliver mainly magazines
and junk mail. It’s been years since a
runner appeared in my doorway breathlessly announcing, “Mr. Luce, it’s a fax
for you! It’s a fax!” Most attorneys
don’t know how to use a Dictaphone,
and virtually all do their own typing. We
have become addicted to Internet e-mail which, for attorneys, turned out to be
the “Killer App.” We get it on our
PCs. We get it on our smartphones. We get it 24/7/365. It is addicting, demanding, disruptive and essential. And now ours was kaput.
Our IT crew, of which I proudly claim to be an “associate
member,” worked valiantly, and largely without sleep, through the night and the
next several days to restore modern order.
By dawn Sean had moved us from Postini to McAfee MX-Logic, so that we could actually access
new incoming mail. We backed-up
everyone’s local Outlook in-box to supplement our regular full mail-server
backups. When, after the controller replacement
arrived, it was discovered the two RAID discs on which our e-mail was stored had
bad sectors, we began a restore from our backup tapes to good discs. Concurrently we prepared our new Exchange
server and brought it online by Friday.
The restoration of other, less critical services,
lagged. Outlook “Contacts” and “Autocomplete”
had to be manually restored on each PC.
Our voicemail-to-e-mail forwarding service did not get back online until
late Friday, and our conference room scheduling was still down as the weekend
arrived. It took about 36 hours for the Internet
root servers to recognize our redirect from Postini to MX-Logic, so the release
of e-mail spooled at Postini was delayed about two days and arrived in a swarm. There were sporadic “bounce-back” issues within
our network and some Citrix spottiness as the “Nerd Herd” worked to get all the
intricate, interlocking electronic gears to mesh again. But we survived.
Lessons Learned
1. Shit Happens.
Systems fail – frequently at the most inconvenient of times. Although the raison d'ĂȘtre of a RAID is its redundancy, a cascading failure such
as we experienced not only brought down our e‑mail, but also the flash-frying
of our controller did not enable us to confirm there were bad sectors on the affected
discs until it was replaced.
We were snake-bitten, for sure, but could we have avoided
this headache? Few law firms have, or
are willing to invest the money required to build completely redundant systems. Nor are they required to by any reasonable standard
of professional care. Comcast, Amazon
and Facebook each experience service interruptions even with formidable redundant
systems. The expense must be balanced
against the likelihood of catastrophic failure. What is required is that attorneys recognize
and plan for the arrival of Murphy and the Gremlins before they appear at our
doorstep.
2. Have a “Plan B.” 9-11 finally drove home to most firms the
wisdom of providing for disaster recovery.
From our “week without e-mail” experience we learned that firms also
need a recovery plan for those disasters falling short of their building being
brought down by terrorists. For example,
our Postini system has, at least until recently, been a fabulous SPAM-catcher,
protecting our network and increasing attorney efficiency. However, Postini’s current iteration has no
provision for e‑mail continuity. As a
result, last week we could log on to Postini and view umpteen Viagra ads and Nigerian
419 advance-fee scams, but could not access incoming e-mail that we actually
wanted. McAfee MX-Logic has a poorer interface,
but provides for e-mail continuity.
Although our IT director acted swiftly to move us to
MX-Logic, any e-mail arriving in the gap between our mail-server failure the
switch-over to MX-Logic was caught in Postini’s inaccessible spool. Moreover, our clients had no way of knowing
our inbound e‑mail was disrupted since it was spooling securely, albeit inaccessibly,
at Postini. In an age where court orders
and pleadings arrive by e-mail, the importance of maintaining e-mail continuity
may have been the most valuable lesson learned.
3. Have a Personal “Plan B.” Attorneys have become utterly dependent
on systems few of them completely understand.
Nevertheless,
A lawyer has a general duty of
competence under RPC 1.1,
which includes the duty “to keep abreast of changes in the law and its
practice.” RPC 1.1 Comment 6. To the
extent that a lawyer uses technology in his or her practice, the lawyer has a
duty to keep informed about the risks associated with that technology and to
take reasonable precautions.
WashingtonState Bar Advisory Opinion No. 2215 (2012) (emphasis added). Similarly, last year ABACommission on Ethics 20/20 Resolution 105(A) recommended this addition to Comment
[6] to Model Rule of Professional Conduct 1.1:
To maintain the requisite knowledge
and skill, a lawyer should keep abreast of changes in the law and its practice,
including the benefits and risks
associated with relevant technology, engage in continuing study and
education and comply with all continuing legal education requirements to which
the lawyer is subject.
(Emphasis in original.)
The Yellow Triangle of Death |
4. Don’t Let the Lawyers Dictate Equipment Replacement
Schedules. As noted, our Exchange
server had been previously scheduled for replacement, but our IT crew postponed
this task to accommodate some lawyers preparing for a Monday trail. Had IT not been so accommodating would our “week
without e-mail” have occurred? We can
never know.
This is tricky office diplomacy as the practice and our
clients come first. However, in a firm
of any size, there will always be some trial or closing, and there will never be an optimally convenient time to take
down a major network component. Accordingly,
administrators must be ready to push-back and support their IT’s replacement
schedule. If attorneys were given absolute
power to veto an upgrade schedule, we’d all still be running DOS 3.1.
5. It’s Still a First World Problem. E-mail is a “disruptive technology,”
in more ways than one. Psychology Today tells us it’s
addictive. Efficiency experts advise us to turn off e‑mail alerts and our cell phones if we don’t
want to experience technology-inducedADD. (BTW, here’s how to turn off the pop-ups and sounds in Outlook.)
From my own experience both are right. That pop-up window and e-mail alert elicit a
Pavlovian response demanding attention which disrupts my concentration and
momentum like a bumper-car collision. When
the e‑mail is really flowing, the default Outlook configuration makes an
average computer light-up and sound like a pinball machine. Lawyers, competitive by nature, find
themselves cast in the roll of the “Bally Table King,” playing electronic
Whack-A-Mole, a/k/a “I can answer that complex legal issue in 1 minute,” to the
detriment of their clients, their professional wellbeing, and possibly their health. There’s a reason Intel and other companies
pilot-tested “e‑mail‑freeFridays” in 2007, and those overcome by an avalanche of electronic
communications declare “e‑mail bankruptcy” – and option attorneys do not enjoy.
Electronic court orders actually requiring immediate
response excluded, losing e-mail for a day or even several days, is still a First World Problem. It can even be a welcome relief.
While most lawyers have willing embraced electronic tethers out of necessity or convenience, the practice of law is not dependent on any machine. Though we must be prepared to deploy our personal “Plan B,” we are not, and must not, allow ourselves to be undone by something as mundane as a computer failure.
I keep thinking of Samuel T. Cogley, the attorney hired to
defend Captain Kirk in the original classic Star
Trek episode, Court Martial. Kirk is ordered to stand trial for allegedly
prematurely jettisoning a research pod, thereby killing Lt. Commander Ben
Finney. Kirk arrives to discover Cogley
has encamped in his quarters and bestrewn it with books.
Books! |
COGLEY: What's the matter? Don't you like books?
KIRK: Oh, I like them fine, but a computer takes less space.
COGLEY: A computer, huh? I got one of these in my
office. Contains all the
precedents. The synthesis of all the
great legal decisions written throughout time.
I never use it.
. . . .
COGLEY: This is where the law is. Not in that homogenized,
pasteurized, synthesizer. Do you want to know the law? The ancient concepts in their own language? Learn the intent of the men who wrote them,
from Moses to the tribunal of Alpha 3? Books.
(You can view this marvelous scene here.) Later, Cogley forcefully argues Kirk should
be entitled to confront his accuser:
COGLEY: The most devastating
witness against my client is not a human being. It's a machine, an information
system. The computer log of the Enterprise.
. . . I speak of rights. A machine has none. A man must.
My client has the right to face his accuser, and if you do not grant him
that right, you have brought us down to the level of the machine. Indeed, you
have elevated that machine above us. I
ask that my motion be granted, and more than that, gentlemen. In the name of
humanity, fading in the shadow of the machine, I demand it. I demand it!
(You can watch Elisha Cook Jr.'s marvelous argument here.)
(You can watch Elisha Cook Jr.'s marvelous argument here.)
While we have not yet fallen under the yoke of our own electronic
creations, there is wisdom in the words of Samuel T. Cogley. The law is not to be found in that “homogenized,
pasteurized, synthesizer,” and we cannot permit our practices to be ground to a
halt when technology fails. From the
perspective of professional risk‑management, the essential counterpart to techno-dependency
is competence and self-reliance. Cultivate
your inner-geek. Develop your personal
“Plan B.” Do these things now, and you will
easily survive your own “week without e-mail.”