The crew weren't using the redundant fuel pumps. They were using the non-redundant fuel line flushing pump as a generator fuel pump, a task it was never designed for and which was not compliant.

That it doesn't restart on restoration of power is by design; you don't want to start flushing your fuel lines when the power returns because this could kill your generators and cause another blackout.

> Main engine shutting of (sic) when water pressure drops

Yeah, this is quite bad. There ought to be an override one can activate in an emergency in order to run the engines to the point of overheating, under the assumption that even destroying the engine will cause less catastrophic consequences than not having propulsion at the time.

> backup generator not even starting in time

There were 5 generators on board. Generators 1 through 4 are the main generators on the HV bus side, and the emergency backup generator is on the LV bus side.

When the incident occurred, the ship was being powered by generators 3 and 4, which were receiving their fuel via the non-redundant fuel line flushing pump. These generators powered the HV bus, which powered the LV bus via a transformer. The emergency backup generator was not running, so the LV bus was only receiving power from the HV bus via 1 transformer.

The incident tripped the circuit breaker for this transformer, disconnecting the HV bus from the LV bus, resulting in the first LV bus blackout. This resulted in main engine shutdown (coolant pump failure) and an automatic emergency backup generator startup.

There is an alternate (backup) set of circuit breakers and transformer that could have energised the LV bus, but the transformer switches were left in the manual position, so this failover did not happen automatically and immediately. There were no company procedures or regulations which required them to be left in the automatic position.

The LV bus also powered the fuel line flushing pump, so this pump failed. As a result, generators 3 and 4 started to fail (being supplied with fuel by a pump which was no longer operating). The electrical management system automatically commanded the start of generator 2 in response to the failing performance of generators 3 and 4.

Generator 1 and generator 2 were fed by the standard fuel pumps, which were available. One main generator is capable of powering the entire ship, so there was no need to start generator 1 as well; this would have just put more load on the HV bus (by having to run the fuel pump for generator 1 as well).

Instead of the automatic transformer failover (which was unavailable), the crew manually closed the same circuit breaker that had already tripped, 1 minute after the first LV bus blackout.

This restored power to the LV bus via the same transformer that was originally powering it, but did not restart the fuel line flushing pump supplying generators 3 and 4 (which were still running, but spinning down because they were being fed fuel via gravity only). This also restored full steering control, but this in itself was inadequate to control the vessel's course without the engine-driven propeller.

The main engine was still offline and takes upwards of half a minute to restart, assuming everyone is in place and ready to do so immediately, which was unlikely.

The emergency backup generator finally started 10 seconds later (25 seconds too late by requirements, 70 seconds after the first LV bus blackout).

Generator 2 had not yet gotten up to speed and connected to the HV bus before generators 3 and 4 disconnected (having exhausted the gravity-fed fuel in the line ahead of the inoperative fuel line flushing pump), resulting in an HV bus blackout and the second LV bus blackout. With only the emergency backup generator running on the LV side, only one-third of steering control was available, but again, this was inadequate without the engine.

3 seconds later, generator 2 connected to the HV bus. 26 seconds later, a crew member manually activated the alternate transformer, restoring power to the LV bus for the second time.

The collision was preventable:

- It is no longer a requirement that the engine automatically shuts down due to a loss of coolant pressure. It was at the time the vessel was constructed, but this was never re-evaluated. If it were, the system may have been tweaked to avoid losing the engine.

- If the transformer switches were left in the automatic position, the LV bus would have switched over to being powered by the second transformer automatically, and the engine coolant pumps and fuel line flushing pump would not have been lost.

- Leaving the emergency backup generator running (instead of in standby configuration) would have kept the LV bus energised after the first transformer tripped, and the engine coolant pumps and fuel line flushing pump would not have been lost.

- If the crew had opted to manually activate the second transformer within about half a minute (twice as fast as they reactivated the first one), and restarted the fuel line flushing pump, a second blackout would have been avoided, and the engine could have been restarted in time to steer away.

This shows the importance of leaving recovery systems armed and regularly training on power transfer procedures. It also illustrates why you shouldn't be running your main generators from a fuel pump which isn't designed for that task. This same pump setup was found on another ship they operated.

Pre-contact everything is about the ship and why it hit anything, post-contact everything is about the bridge and why it collapsed. The ship part of the investigation wouldn't look significantly different if the bridge had remained (mostly) intact, or if the ship had run aground inside the harbor instead.

No, we are not talking a little paint-swapping.

The spring terminals should also be designed to have a secondary latch on this type on (what should be) rugged installation.

Finally, critical circuits should be designed to detect open connections, and act accordingly. A single hardware<->software design for this could be a module to apply across all such wiring inputs/outputs. This is simple and cheap enough to do these days.

A manual tug-test on the physical would be advisable when installing, to check the spring terminal has gripped the conductor when latched.