Management Summary
Business-critical knowledge must not depend on individual people.
If a system administrator or another key person becomes unexpectedly unavailable, missing documentation, access credentials and deputies can very quickly turn into a significant operational risk.
If a system administrator or another key person becomes unexpectedly unavailable, missing documentation, access credentials and deputies can very quickly turn into a significant operational risk.
Companies should therefore identify critical person-dependent knowledge, document operational processes, properly train deputies, and secure administrative access and emergency information independently. The objective is not to document as much as possible, but to ensure that operations can continue even without a particular key person.
The simplest test: Imagine that, starting tomorrow, a key person could not be reached for three months. Could you keep the business running without having to ask that person any questions? If not, action is required.
You'll have to ask our system administrator
You hear this or a similar sentence surprisingly often in companies.
“Only Mr Meyer knows that.”
“Our accountant set that up back then.”
“Ms Müller is responsible for that.”
Or simply: “No idea, IT always handles that.”
“Our accountant set that up back then.”
“Ms Müller is responsible for that.”
Or simply: “No idea, IT always handles that.”
At first glance, there is nothing unusual about this. Division of labour is part of every company. No one can or needs to know everything. It only becomes problematic when the full answer is actually:
“Only our system administrator knows that.”
Because then we are no longer talking merely about specialisation. We are talking about a business risk.
What happens when you can no longer ask him?
As long as the person concerned turns up for work every morning, this dependency is hardly noticeable.
The system administrator knows the servers. He knows where the passwords are stored. He knows the internet provider, the firewall, the backup system and the peculiarities of the applications. He knows why some program has been started every night at 02:15 for many years – and probably also what happens if it fails to start. Everything works. So there seems to be no problem.
Until the employee becomes unavailable. Then that seemingly harmless sentence suddenly takes on a very different meaning:
“You'll have to ask our system administrator.”
Except: You can no longer ask him.
The system administrator as living documentation
Especially in small and medium-sized companies, IT landscapes have often evolved over many years. At some point, an employee started looking after a few computers in addition to his normal duties. Then a server was added. And much more followed.
Eventually, that one employee possesses knowledge about the company that goes far beyond actual system administration.
For example, he knows:
- which server performs which task
- which applications depend on one another
- which technical users are used
- where licences are managed
- which certificates need to be renewed
- how the backup works
- which systems communicate with one another at night
- which external service provider needs to be called for which fault
- and which special points need to be considered for month-end, quarter-end or year-end processing
This knowledge is valuable. The system administrator is not the problem. On the contrary: an employee with this wealth of experience is extraordinarily valuable to a company. The problem arises when this knowledge exists only in that person's head.
The company may then have functioning systems – but possibly not full control over those systems.
The problem is not limited to IT
What applies to the system administrator can be found in almost every area of a company. One employee is the only person who knows how the monthly VAT return is prepared. An administrator knows all the special arrangements for the most important customers. The production manager knows what to do when a particular machine fault occurs. Another employee knows about agreements with a supplier that were made twelve years ago and never properly documented.
And almost every company has at least one employee about whom people say:
“He simply knows everything.”
As a compliment, that is certainly meant positively. From a risk-management perspective, however, this statement should immediately set off an alarm. The more important one person's knowledge is to the continued operation of the company, the more important it is to make that knowledge available to the company itself.
This does not mean trying to make good employees replaceable. It means something quite different:
The company must remain capable of acting even if a key person suddenly becomes unavailable.
A true story
Some years ago, we were confronted with a situation that could hardly have illustrated more clearly what missing documentation can mean. It involved a Swiss SME with around 200 employees. The accountant felt unwell, lay down for a moment and never came back.
The accountant, however, was far more than just an accountant. He was an employee of the old school. He had been with the company for many years, was reliable and experienced, and had extraordinary knowledge of the business. He knew the accounting. He knew the payroll. He took care of numerous administrative processes. And on top of that, he was also responsible for significant parts of the IT environment.
In short: He knew how the company worked.
The problem was: None of it was documented.
There were processes, but no runbooks. There were programs, but hardly any descriptions. There were administrative deadlines and dependencies that had always been handled reliably – mainly because this one employee knew what had to be done and when. From one moment to the next, that knowledge was no longer available.
Operations must continue
Our company was able to provide four people at very short notice to maintain an emergency operation. This was by no means simply a matter of getting a few computers running again. Among other things, payroll had to continue; around 200 employees quite rightly expected to receive their salaries. Payments had to be made. Administrative deadlines had to be met. Data had to be processed. Business processes could not simply stop.
So the reconstruction began. We analysed programs, files, printouts, filing systems, schedules and existing documents. We spoke with employees and tried to establish who knew which parts of a process. Piece by piece, an overall picture began to emerge again. We created runbooks for the different areas. The reconstructed processes were documented and handed over to the responsible staff together with the information they needed.
After about two months, we were able to hand over a properly reorganised company administration. Operations were once again based on a much broader foundation of knowledge.
The company was lucky, because it could have been considerably worse.
If, for example, essential access credentials had been missing, external systems had no longer been reachable, or central processing steps had been impossible to reconstruct, the damage could have been significantly greater.
One very simple principle can be derived from this experience: Knowledge that is necessary to operate a company must never belong exclusively to a single person.
That applies to the accountant. To the production manager. To the managing director. And, of course, to the system administrator.
Documentation is not bureaucracy
When documentation is requested, another sentence is often heard: “We don't have time for that.”
That is understandable. Documentation takes time.
That is understandable. Documentation takes time.
But perhaps the question should be asked differently: How much time do you have if the person concerned is no longer available tomorrow?
Suddenly, very different questions arise:
- Which provider actually operates our internet connection?
- Under whose name was our domain registered?
- Who has administrative access to Microsoft 365?
- How do we access the firewall?
- Where is the latest functioning backup?
- Who can administer our ERP system?
- Which process has to be started at month-end?
- Which certificates are due to expire soon?
- Which service providers need to be called in the event of a fault?
- And what exactly does that one server do that nobody has dared to touch for years?
Good documentation answers such questions before an emergency occurs, not only after it has happened.
Documentation has another benefit as well: anyone who describes a process properly is forced to question it. Unnecessary dependencies, unclear responsibilities or technical weaknesses often only become visible at that point.
Documentation is therefore not merely a safeguard. It is also an opportunity to review and improve existing processes.
That is not unnecessary bureaucracy. It is part of professional corporate management.
But documentation alone is not enough
A folder labelled “IT documentation” is not yet emergency preparedness. Documentation must be up to date. It must be understandable and it must be available in an emergency.
A perfect recovery guide stored exclusively on the server whose total failure has just occurred is of only limited practical value. Passwords should, of course, not be stored indiscriminately in documentation either. Critical access credentials require controlled password management and defined emergency access.
At the same time, not every piece of information needs to be accessible to every employee. Passwords, trade secrets and other sensitive information need appropriate protection. Work instructions and process descriptions, on the other hand, should be available to the people who work with them and are responsible for them. For particularly protected information, it must nevertheless be ensured that authorised people can access it in an emergency.
Security and availability are not opposites. Both have to be organised.
The actual objective is therefore not: “We must document as much as possible.”
Rather: An authorised deputy must be able, with the available information, to keep operations running or to bring in the right specialists in a targeted manner.
Take the three-month test
There is a simple thought experiment for executive management and middle management. Imagine this: your system administrator finishes work this evening and cannot be reached for three months. Not by telephone or by any other means.
What happens tomorrow morning?
- Can someone access all the necessary systems?
- Does someone know about the currently open issues?
- Does someone know which tasks have to be performed daily, weekly, monthly and annually?
- Can users be created, changed and disabled?
- Can a backup be restored?
- Are the external contacts known?
- Does someone know when licences, certificates and contracts expire?
- Can a new system administrator understand how the environment is structured?
If you have to answer even one of these questions with “No”, action is required. It becomes even clearer if the answer is: “Our system administrator probably knows.”
To-do list: the most important points
- Identify key people
Determine, in all important areas of the company, which knowledge required for operations is held by only one person. A useful question is: “What would stop working if this person were unavailable for three months starting tomorrow?” - Identify critical processes
Document the processes that are indispensable to business operations. These may include payroll, payment transactions, invoicing, production, purchasing, backups and central IT processing. - Define deputies
At least one second person should be designated for every critical task. A deputy shown on the organisation chart is not enough. That person must have the necessary knowledge and permissions. - Document the system landscape
Servers, applications, networks, cloud services, interfaces, and internal and external connections must be described in a comprehensible way. It should also be clear which systems depend on one another and which external services are required for operations. - Secure administrative access
No business-critical access must depend exclusively on one employee's personal account or knowledge. This applies particularly to providers, domains, cloud services, firewalls and central applications. - Manage passwords professionally
Critical passwords do not belong on Post-it notes or in unencrypted Excel spreadsheets. Use an appropriate password-management solution and define controlled emergency access. - Record external partners and contracts
Document relevant providers, software vendors, maintenance partners and specialists. This includes contacts, customer numbers, contract information, support channels and, where applicable, agreed response times. - Create operating runbooks
Recurring and critical tasks should be described in such a way that a qualified deputy can follow them. Daily, month-end and year-end processing, as well as typical fault situations, are particularly important. - Test backup and recovery
“We make a backup every day” is not a sufficient statement. What matters is whether that backup can actually be restored. Regular restore testing is part of emergency preparedness. - Keep emergency information independently available
The most important information must remain accessible even if the network, servers or central cloud services are unavailable. Data protection and access control must, of course, remain guaranteed. - Review documentation regularly
Outdated documentation can be more dangerous than having no documentation at all because it creates a false sense of security. Responsibilities and review intervals should therefore be clearly defined. - Actually practise the emergency scenario
The most meaningful test is simple: the system administrator must not help during an exercise. Give a qualified deputy a specific task and check whether it can be completed using only the available information. Every necessary question reveals a gap in documentation or knowledge.
Safeguarding knowledge is a management responsibility
Responsibility for this issue does not lie with the system administrator alone. Of course, a system administrator must document the systems properly. But whether a company systematically safeguards critical knowledge is ultimately a management decision. If a system administrator has been looking after all systems alone for years while spending every free minute on user problems, projects and incidents, it should come as no surprise if the documentation is incomplete.
Documentation takes time. Knowledge transfer takes time. Training a deputy also takes time. But this time is not unproductive.
The time invested is an investment in the company's operational resilience.
A deputy also needs to actually deputise from time to time. Someone who has been shown on the organisation chart as “the emergency deputy” for five years, but has never personally restored a backup, administered a user or carried out month-end processing, is only very partially prepared for a real emergency. A good system administrator should therefore not be judged by how many tasks only he can perform.
On the contrary: A professional system administrator creates structures that allow operations to continue even without his immediate presence.
That does not make him less important. It demonstrates his professionalism.
From personal knowledge to corporate knowledge
No company can document all the knowledge of its employees. That would be neither possible nor useful. But a clear line must be drawn when it comes to business-critical knowledge.
The key question is: Does the company need this knowledge in order to continue operating?
If the answer is yes, then this knowledge must be made available to the company in an appropriate form.
Perhaps you yourself are the person who carries a significant part of the company's knowledge in your head. Then the same question applies to you: Can operations continue if you become unavailable for an extended period starting tomorrow?
This question does not arise only in an emergency. During a business succession, a sale or a handover, it also becomes very clear how much knowledge actually belongs to the company – and how much exists only in the heads of individual people. Good documentation therefore does more than provide security. It also makes a future handover of the company easier.
External person-dependent knowledge
The same applies to knowledge held by external partners – by individual specialists just as much as by large solution providers. Especially when projects are outsourced, it should therefore be clearly defined at the procurement stage which documentation must be created, what quality it must meet and when it must be handed over. These requirements should not be reviewed only at the end of the project, but throughout all project phases.
Even large providers do not automatically protect a company against dependencies on individuals. If an important specialist leaves a project team, valuable knowledge can be lost or may have to be rebuilt at considerable expense. This becomes particularly problematic when important decisions, configurations or technical relationships are known only to individual project team members.
The value of good documentation becomes particularly apparent when a provider is changed later on. A properly documented solution can be taken over and continued by a new partner. Without this basis, it is first necessary to reconstruct what was originally built and why certain decisions were made. That costs time and money and also increases operational risk.
Documentation should therefore be a binding part of project acceptance.
Critical components should only be considered complete once the associated documentation is complete, up to date and comprehensible. It is also sensible to link relevant milestones and payment approvals contractually to the handover of the agreed documentation. It is not enough simply to check whether a document exists and has been formally approved. The decisive question is whether a qualified person can actually understand, operate and, if necessary, take over the solution using those documents.
If the company itself does not have the necessary expertise for this assessment, it may be advisable to have the documentation reviewed by an independent specialist. The review should not merely check whether all required documents and signatures are present, but above all whether the content is complete, understandable and technically usable.
The same principle therefore applies to externally provided services: At the end, the knowledge required for operations must be available to the company – and must not remain with the project team or the supplier.
Ask your system administrator
Perhaps you really should ask your system administrator something. But this time, not: “How does this system work?”
Ask him: “What would happen here if you could not be reached for three months starting tomorrow?”
A good system administrator will probably be able to name a few points immediately. Listen carefully. That conversation may be the starting point for one of the most important sets of documentation in your company.
And if, when one of the next questions is asked, someone answers again: “You'll have to ask our system administrator”, your follow-up question should be: “And where is it documented in case we can't ask him one day?”
Further reading
If you would like to explore business continuity, emergency preparedness and the safeguarding of business-critical processes in greater depth, the following sources provide further information:
- BSI – BSI Standard 200-4: Business Continuity Management
A practice-oriented standard for establishing and operating a Business Continuity Management System.
www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Grundschutz/BSI_Standards/standard_200_4.pdf
Comment: The current BSI Standard 200-4 is available in German only. - ISO – ISO 22301:2019: Business Continuity Management Systems
International standard for establishing, operating and continually improving a Business Continuity Management System.
www.iso.org/standard/75106.html - National Cyber Security Centre NCSC – Emergency planning for Swiss SMEs
Practical Swiss guidance for preparing small and medium-sized enterprises for cyber and IT emergencies.
www.bacs.admin.ch/en/26-notfallkonzept-en
This article is a translation of the German original and was created using artificial intelligence
German original: "Da müssen Sie unseren Administrator fragen"