Automation in overdrive: engineering a revolution
Written by Joe Pontin.
Could a leap forward in automation help transform safety on railways – by dramatically improving the quality, scale and availability of track monitoring data? In part 2 of our article tracing the rise of automation in railways, we explore the problems OBC engineers solved to deliver their automated intelligent video system, AIVR
Scaling up: AIVR in detail
A two-minute stroll from the old Bristol Temple Meads station – built by the great engineer Isambard Kingdom Brunel nearly two centuries ago – brings you to the lab of tech firm One Big Circle (OBC). On this May morning, sunlight streams through the high windows of the lab, whose staff are quietly assembling cameras and sensors in protective housings sprayed British Rail signal yellow.
One Big Circle’s engineering team assemble monitoring devices in the Bristol lab.
These devices are the latest to join a growing throng of automated lookouts that keep watch on our railways. Perched on trains and tracksides, they will form part of the Automated Intelligent Video Review (AIVR) system, as its makers call it. AIVR deploys the latest tech – including machine learning, artificial intelligence and sophisticated Cloud communications – to spot potential faults on tracks and trains, and help raise the alarm before any harm is done.
Once the bolts are tightened on the housings and the glass is given one final polish, they are ready to go. From here OBC engineers take them to Derby or elsewhere for fitting on trains; then they go off to live their life on the rail network.
Once out on Britain’s railways, AIVR devices are remarkably autonomous. Engineers clean them every now and then, but otherwise, all being well, they are seldom seen.
With this level of automation, AIVR represents a huge leap forward in track inspection tech (the history of which we recounted in part one of this article). But HOW does AIVR work autonomously? This second part of our article drills down into the detail, looking at the challenges OBC’s engineers faced in delivering a reliable, automated system, and how they resolved them.
We start with a look at some of the broad principles behind AIVR.
Automation allows vast increases in scale. Photo by eAlisa.
Automation the OBC way
Scaling up
“The errors which arise from the absence of facts are far more numerous and more durable than those which result from unsound reasoning respecting true data.” So argued Charles Babbage, the ‘Father of computing’ (and as we learned in part one, pioneer of track-testing), back in 1832.**
From Babbage’s observation, you might reasonably infer that the more data, the better your decisions are likely to be.
This is the basic principle of Big Data, one of the fundamental components of the Fourth Industrial Revolution (4ID; see also part one). And it’s very much the approach at OBC, too. Co-founder Ian Packer confirms: “Our approach is to capture more. You get more authority from the scale of data.”
Ian points out that OBC was not the first to provide the rail industry with a track-monitoring system. In 2019 – the year OBC first got involved in railways – other firms were already providing high-quality data – but, crucially in relatively limited quantities that “narrowed the scope of understanding”, says Ian.
Something was missing: scale.
Your laptop is a portal to a rapidly expanding universe of information.
As we discussed in part one, if automation has a superpower, it is scale. So a properly automated system should transform the scale of data available. Add to that complete accessibility, and if OBC could nail this, there would be a huge win: rail engineers of all disciplines would soon be able to see the whole picture, rather than fragments of it.
Freeing devices from manual constraints
From the beginning, says OBC software engineer Ed Torbett, the approach was to strip away as many physical dependencies as possible. “At each step we have moved further and further away from manual input,” says Ed.
For example, OBC’s engineers began to consider whether the rail industry might be able to abandon the movement of data via ‘wires and wheels’ (see part 1), and instead move data (mostly video) at scale – via the Cloud. (Much more on this below.)
Most importantly, AIVR was always designed as an unattended system. “The idea of having to have someone monitor the device as it runs and make manual adjustments to the recordings as part of the shift was never something we considered. We needed a system we engineers could drop in place, then go about our day,” says Ed.
Changing roles
As we discussed briefly in part 1, in every industry, growing automation almost always leads to a change in working roles for some people, and in a period of transition, this can cause anxiety.
This is understandably a sensitive issue, and will affect individuals in different ways. Through its history, automation has displaced certain roles. It also creates new ones, and may transform existing jobs in ways that can make the work more rewarding.
Far from eliminating the need for skilled engineers, automation increases demand for technical expertise. While automation takes on repetitive tasks, engineers are freer to use their real skills – as problem-solvers. For example, in analysing and interpreting data, system maintenance, programming, and the design and oversight of automated systems.
In an industry like rail, automation delivers real safety benefits to rail workers, as well.
OBC’s first track monitoring device was AIVR Go, mounted on a train windscreen.
In 2019, OBC’s engineers placed the first AIVR cameras on train windscreens, and since then the firm has rolled out a sequence of increasingly sophisticated rail, trackside and train monitoring devices under the AIVR umbrella.
In successfully creating this new automated system, OBC’s engineers faced a series of difficulties posed by the unique environment of the rail industry.
Automation is hard. Why?
Creating automated devices that accurately measure the world around them is one thing; doing this in the railway environment adds particular challenges. “Rail is one of the biggest analogue systems out there – literally metal grinding on metal – and we are trying to efficiently represent that world digitally,” says Barnaby Kent, co-founder of OBC. Most of these devices must work on trains that are whizzing around the countryside at up to 125mph, in all weathers. As Barnaby says, “Automation is hard enough; now try putting it on a 100-tonne steel horse.”
Saving lives: why rail monitoring matters
Plain and simple, effective track monitoring helps keeps people safe on the railways. As we explored in part one, a damaged track has the potential to derail a train, causing a serious accident. But spotting signs of damage on the UK’s 20,000 miles of track is a huge task. In the two centuries since the first passenger stepped on a steam train, engineers have devised more and more sophisticated ways to monitor the condition of tracks (see part one). Then in 2019, Bristol tech firm One Big Circle launched AIVR – aiming to revolutionise the quality, quantity and availability of data on the condition of tracks, trains and trackside environments, so all relevant engineering disciplines would have the accurate and up-to-date information they need.
Harsh conditions
AIVR devices are loyal soldiers, but their life on the network gives them quite a hammering: flung into the teeth of a gale, plastered with grime, even electrocuted, they must recover and report for duty without a grumble.
Changing weather can profoundly transform optical conditions for cameras – especially rain [read about this in detail]. Then the train itself can be a hostile environment. Sitting inside a carriage, serenely gliding through the Cambridgeshire Fens, it’s easy to forget that these heavy steel behemoths contain powerful engines, and hefty steel wheels mounted on steel bogies. Both cause devices to vibrate. Modern trains are often powered by electricity, which can disturb or even burn out nearby devices. And of course railways are also grubby places that fling dirt and dust in the eyes of OBC’s little lookouts.
All this means devices must be well protected; hence that smart yellow housing. But they depend on humans for cleaning and maintenance, too. (Look out for our articles on how OBC protects its cameras, and on the problem of space on trains.)
Diverse environments
As we’ve mentioned, changing weather can radically alter the environment in which monitoring devices work. But diversity is built into the rail environment in other ways, too.
OBC’s autonomous devices are precision instruments – they are used to discover and precisely locate damage to tracks, so any variation in their physical location on trains can be consequential. Unfortunately, though, trains differ in design and configuration, so you can’t always put automated devices in the same place on each train. That means, for example, that camera positions and angles vary.
On top of that, some trains are designed to power off at the end of a journey; others will maintain a power supply. And some types of location, such as tunnels, are especially challenging places – they disrupt communications, including positioning systems.
When life throws curve balls like this, the device must be able to adapt and recover.
As OBC co-founder Emily Kent puts it, “It’s an incredibly diverse environment, so when we first began work on railway monitoring, we needed an adaptable system that was intelligent enough to perform consistently despite the many variables at play.”
Use Case 1: Forward Facing Cameras
AIVR’s forward-facing cameras capture a uniquely complete picture of the network, including this view of London’s Battersea Power Station from Grosvenor Bridge.
After the fatal accident at Margam in 2019, Red Zone Working – that is, working on or near the line while the railway is open to traffic – has effectively been banned, for safety reasons (read more in part one of this article). This makes physical inspection of tracks more problematic, as it requires line closures, and/or night-time inspections.
Fortunately, rail engineers can now keep in touch with the rail network using AIVR. The system’s Forward-Facing Cameras are out on the tracks all day, providing real-time HD video of the railways. Transmitted via the Cloud, the footage allows AIVR’s 10,000 users – including engineers of all disciplines – to view track conditions from the safety of their computers.
Engineers can identify debris, encroaching trees or bushes, and keep an eye on the state of signals, bridges, tunnels, and viaducts.
It’s a safe and cost-effective solution for railway monitoring that keeps trains running, and helps engineers to identify problems remotely – allowing any necessary line closures to be devoted to repairs.
Essential capabilities of automated devices
Regardless of the challenges of the rail environment, any network of automated monitoring devices will need to deliver some essential functions if it is to be of any use. We’ll sum these up in brief first, then explore some of them in a bit more detail afterwards.
Real world interfaces
Remote devices must include high quality cameras and other sensors to gather video footage and other data.
Automated recording
The devices must have the autonomy to turn recordings on and off as necessary.
Communications
They must be able to communicate remotely. When communications are temporarily lost, the device must be able to recover.
Positioning
Devices must be able to log their position as they move through the rail network. (For more on this, see our forthcoming article on positioning.)
Health checks
They should be capable of send back diagnostics – information both about their own condition, to help inform maintenance, and on the status of any tasks they are performing.
Coordination
Remote devices must be subject to control and management by an intelligent application or ‘agent’, ensuring the right quality, quantity and coverage of data is collected.
*****
So now we know a bit about the unique challenges for devices operating in the rail environment, and we understand what we need from automated devices if they are to work effectively.
Below, we’ll explore three of the essential capabilities we’ve just mentioned – automated recording, communications and health checks. (Positioning will be covered in more detail in a separate article.) After that, we’ll look at the coordination of remote devices in a bit more detail, in the section titled Coordination: How to orchestrate a symphony of data.
When it comes to maintaining power supply, unnecessary switches can be a real turn off.
1 Self-starters: automated recording
What was the most fundamental requirement of automation? “The device has got to live on its own,” says Barnaby. That meant starting with devices that could automatically turn themselves on and off.
Track monitoring devices made by other brands typically need to be switched on at the start of every day, and keep recording until switched off, “like a dashcam,” says OBC software engineer Ed Torbett. “That would be simple,” he says, “but then we would have an awful lot of footage of the train sat around not doing anything.”
Problem-solving
Movement-activated recording
The solution – “probably one of the first things that we brought under the device’s control”, says Ed – was to trigger recordings based on movement. When the train starts moving, the choice to start recording is made by the device, rather than by a human or an external system. The device makes this decision based on connected positioning systems – for example, GPS, or the tachometer used in some AIVR systems. [For more on this, look out for our forthcoming article on positioning.]
Removing the on/off switch
Despite this built-in autonomy, early versions of AIVR included physical on-off switches – effectively a manual over-ride allowing the device to be powered down.
The on/off switch was not really needed – as the device would turn *itself* on and off. Understandably, well-meaning train crews sometimes assumed that the devices actually required switching off at the end of a journey. Unfortunately, once switched off, the device would not automatically turn itself on again.
So in later versions of AIVR, OBC simply removed the on/off switch. “If it has power, it should be operating,” says Ed.
Pre-record buffering
As we mentioned, when the device’s positioning systems report movement, the device ‘turns on’ and begins to record. OBC engineers soon encountered a problem: a lag between the train moving and the positioning system first reporting movement. In that lag, the train might have travelled some distance without recording – leaving a stretch of track unchecked.
The solution was a “pre-record buffer”, so that the device continuously records, even when it thinks it is stationary, but only retains the last few minutes of this footage, deleting anything older on a rolling basis. It does this until its positioning systems report movement; whereupon the buffer footage is retained.
So as long as it has power, the device doesn’t really switch off at all – in its otherwise dormant state, it continues buffering until it senses movement.
OBC engineers added an override to allow rail engineers to shoot footage when exploring tunnels on foot.
Overrides
In some circumstances, engineers wanted AIVR to record without the trigger from positioning data. For example, sometimes teams want to use body-worn AIVR cameras to manually inspect tunnels, where there is no GPS signal to register movement and trigger recording. “We need to *force* devices to record when people are doing stuff in tunnels,” says Ed. This involved adding in “a bunch of features to allow our decisions to be overridden”. A companion app allows the operator to manually start or stop recording if required.
A helping hand: temporary devices
OBC operates some devices that **do** need significant manual support. Housed in T-boxes – the bright yellow metal containers – these units are attached to brackets on the train front on a short-term basis and have a limited battery life of six to eight hours. Even in this case, though, “we do our best to preserve that battery by being intelligent about how the system operates – such as automatically turning off the device at night,” says Ed.
Battery back-up
AIVR devices transmit large quantities of data via the Cloud (much more on this below). This often takes longer to upload to the Cloud than the journey itself, so when the train comes to a halt, a significant backlog of data may still be awaiting upload. If the train power is switched off soon after journey’s end, the download is paused until the train powers up again, often the following day.
Batteries now give AIVR devices a chance to finish transmitting footage even after the train has powered down.
Ed says OBC is now introducing a way to avoid this delay by providing enough power to AIVR devices to complete these uploads, even without the train’s power supply – using batteries. OBC calls this solution AIVR Ready.
“With AIVR Ready, we want to see how long we can continue operating, even after the power has gone. This is good for high-precision systems such as AIVR Focus, which generate large amounts of data.”
When the train power comes back on, AIVR Ready’s batteries recharge.
Use Case 2: Forward Facing Cameras
Forward Facing Cameras have delivered some unexpected benefits to rail engineers. “It has proved way more flexible that anyone here envisioned it to be,” says One Big Circle engineering manager Jasper Roseland.
For example, they are helping to partially automate the process of track design.
When complicated junctions need replacing, full track surveys remain necessary – but images from Forward Facing Cameras can provide a slew of useful extra information.
Using digital measurement tools – calibrated from the track gauge – designers can make precise measurements of the tracks and trackside environment remotely, saving time and costs.
Like all good kids, AIVR devices like to keep in touch with their family.
2 AIVR phone home: communications
While AIVR devices are designed to function out in the real world without help for as long as possible, like all good parents, the OBC engineers like to keep in touch. Apart from anything, without reliable communications, AIVR devices cannot do their job.
Many trains are now fitted with multiple AIVR devices, each with its own antenna transmitting data to the Cloud.
Engineers and remote devices stay in contact via a 4G/5G connection to the Cloud: AIVR sensors and cameras are wired to a hub inside the train, where computer gear is racked and housed in bespoke metal enclosures; from there data is communicated through the Cloud to the outside world via antennas, often attached to the train roof. There may be up to six antennas on one train, each associated with a different device.
(How are the communications between the Cloud and AIVR’s remote devices managed? You can read about this below, in the section titled Coordination: How to orchestrate a symphony of data.)
Problem-solving
Breaks in communication
Inevitably, AIVR devices go quiet sometimes, for a variety of reasons – the train might be switched off, or in an area of poor signal, for example. The device must be able to recover from this break in communications and ensure that no data is lost.
According to Ed, AIVR devices are autonomous enough to press on when comms go down. “The device just gets on with it,” he says. “It’s always autonomous – it doesn’t operate differently whether it has comms or not.”
For example, when a significant event – such as an emergency stop – occurs while communications are down, the device simply waits for the opportunity to report it. “Event reporting is entirely managed by the device,” says Ed. Even if the device can’t talk to the outside world for a while, it will record the event, store any data, and then upload it when it can. “We are not going to lose footage just because we don’t currently have a network connection – it’s not like a remote CCTV camera, where you’re either connected or you’re not.”
AIVR’s strategy for data transmission resembles bee behaviour. Photo by Simon Kadula.
AIVR Swarm
When signal is limited, could there be a way of making the best use of any connection that remains? OBC engineers noticed that on any given train at any given moment, one device might be successfully transmitting data through its antenna, while another device had gone quiet.
Remember, each of the antennas on the train, up to six of them, is associated primarily with a single monitoring device, and each of these monitoring devices is wired to a central hub inside the train. In geographical areas where signal is inconsistent, the antennas may vary widely in performance, as environmental factors influence the quality of the signal. It may be just the same at home, where your mobile phone signal in the bathroom is better than the one in the kitchen. To give a concrete example in the case of trains, the antenna on the train roof – associated with the system monitoring the overhead lines – might lose signal altogether; meanwhile the antenna on the train front, associated with the forward-facing camera, may be working relatively well.
For times like these, OBC came up with a new system that allows all its devices to send their data using the antennas with the least congested connections.
So when signal goes down in five antennas, but remains less congested in the sixth, then the data from all devices flows through the train’s central hub to that particular antenna, for transmission to the cloud.
Imagine a beehive; the swarm of bees inside representing the data generated by devices on the train. The hive normally has six exits; but one day, five of the six exits become temporarily blocked. So all the bees head to the same exit. That’s why OBC calls this system AIVR Swarm.
3 Health checks
So what happens when things go wrong, and devices need fixing? Some problems may be diagnosed remotely, but require manual intervention to put them right: “Hard drives fail, components work loose, maybe a system on the train gets accidentally unplugged.” But other problems can be fixed without ever even touching a device.
Problem-solving
Self-healing
Remote devices can diagnose and treat some of their own ailments. For example, the modem may get into a state where it can’t do anything, and will automatically re-boot. OBC calls these decisions by remote devices “Interventions”.
Diagnostics
Sometimes rebooting isn’t enough. Fortunately, all OBC devices regularly phone home and report status – in fact, “they update us every 60 seconds,” says Ed, allowing OBC engineers to identify problems remotely, and “get in and maintain our devices and keep them healthy”.
Remote devices supply diagnostic information not only about their health, but also the ongoing performance of their functions. For example, they report on any footage they have yet to upload.
You keep them on a short leash, I say. “Who knows what they’d get up to if we didn’t?” smiles Ed.
Remote maintenance
When devices hit problems, some can be sorted by engineers from their laptops. “A lot of our kit needs to be remotely maintainable,” says Ed. He gives an example: those all-important positioning systems occasionally need recalibrating. (These positioning systems allow OBC to give accurate coordinates to the rail engineers who locate and fix track faults.)
Secure comms
Every device is securely connected to a maintenance VPN (virtual private network), which allows OBC both to remotely monitor devices and to connect to them for remote maintenance. The VPN uses industry standard encryption and secure connections to allow OBC to access devices safely and securely without exposing them to the risk of external connections from anywhere else.
Among other positioning systems, OBC uses a tachometer to precisely locate the train. The tachometer works by counting the revolutions of the train’s wheels, and multiplying that by the wheel circumference.
Train wheels may be tough, but they do a tough job – and eventually get worn down. Photo by Peter Moulton.
The snag is that a wheel’s size may change. Over thousands of miles bearing heavy trains over steel rails, the wheel will wear down. This wear is usually uneven, so as part of their maintenance regime, worn wheels are occasionally ground to restore a smooth surface. Of course, this slightly reduces their diameter. Over the tens of thousands of revolutions the wheel may make in a journey, the inaccuracy in tachometer readings grows ever wider.
Fortunately, the rail engineers who grind the wheels routinely report their new circumference, and thanks to remote maintenance, OBC engineers can use these revised measurements to remotely recalibrate their remote devices.
Other remote work includes rebooting systems, checking and reestablishing connections to a new sensor, and updating software to enable new capabilities.
Use Case 3: AIVR Focus
AIVR Focus is a line-scanning camera mounted beneath the train, which records the condition of the track in minute detail. Literally every millimetre of track is photographed – even when the train is travelling at more than 100mph.
OBC uses machine learning models to scrutinise tracks in minute detail.
The images captured by AIVR Focus allow engineers to make virtual fingertip searches for track faults from their computers, then provide the information necessary for rail engineers to locate and repair the track.
For example, using machine learning models, OBC engineers recently spotted some damage to a track in the Midlands. They reported the fault to Network Rail, and within 36 hours rail engineers had visited the location and repaired the track.
AIVR Focus provides other benefits, too. When Network Rail inspectors find defects on the track, such as a missing bolt on a joint, engineers can call up data from the AIVR archive to look back at the history of the fault. This helps rail engineers learn how faults occur and over what timescales they develop.
The same data can also be used to improve the algorithms used to spot track faults remotely. Those images of the early stages of joint damage help train AIVR’s machine learning models to recognise the signs of similar damage elsewhere on the network.
To keep all elements in harmony, a complex system with multiple inputs requires coordination – like an orchestra conductor.
4 Coordination: How to orchestrate a symphony of data
The last of the essential tasks of monitoring devices, listed near the top of this article, was related to coordination – they must be subject to control and management by an intelligent application or ‘Agent’, ensuring the right quality, quantity and coverage of data is collected.
So far we have looked mainly at the ways automation has helped OBC devices function in situ. This final section addresses the ways automation helps AIVR’s many component parts work as a fluid, interactive system.
This is important because with hundreds of devices feeding data into AIVR’s data centre in the Cloud, organisation is not easy. “Complex system orchestration is big, messy stuff,” says Barnaby. “We are bringing order to this process in an automated way.”
The Cloud: benefits and limitations
The Cloud seems ethereal, but in reality depends on the support of a system of data centres. Photo by miss irine.
‘The Cloud’ deliberately invokes the idea of a virtual space, but of course in one sense it is rooted in the physical – it comprises a network of remote servers that communicate via the internet, providing essential computing services such as data storage and processing power.
The Cloud is capable of handling vast amounts of data, allowing tech firms such as OBC the flexibility to make sudden leaps in the scale of data they are handling. Create a data centre accessible via the internet, and your remote devices can interact with the Cloud.
What we mean by ‘The Cloud’
For the sake of simplicity, we occasionally refer in this article to devices uploading data to ‘the Cloud’. This might imply that data gets pumped into one amorphous mass shared by all Cloud users. Of course that’s not the case. It would be more accurate to say that remote devices send data via the Cloud to a centralised data centre. In the case of AIVR systems, this data centre is called, appropriately, AIVR Cloud.
With this superpower, the Cloud was the obvious way to free data gathered by track monitoring devices from the constraints of wires and wheels, when data was downloaded from computers on monitoring trains to portable hard drives and couriered to its destination (see part one). With the Cloud, that data could, in theory, be beamed rapidly from devices to engineers from all relevant disciplines.
Video had been used to monitor tracks before AIVR, but “using *video* on the Cloud,” says Ed, “was the differentiator” – the thing that allowed AIVR to offer something new and exciting to the rail industry. (Ed points out that OBC engineers’ previous use of the Cloud – for both transport infrastructure projects and a sport video sharing service – “had given us the experience to know how to run an application that sits in the Cloud, and to take advantage of the huge scalability the Cloud can provide.”)
Huge influxes of data can cause backlogs, just like queuing traffic at rush hour. Photo by Alexandra Gl.
Ed and his colleagues at OBC well knew that while the Cloud offered huge advantages, there were challenges, too. That’s because the Cloud’s capacity to cope with vast influxes of data was limited – when that data was transferred at standard internet rates. High volumes of data uploaded this way to the Cloud can cause long backlogs of information. For example, since AIVR launched in 2019, its devices have generated around 1.7 petabytes of data. “On a normal home internet connection, that would take three to six years to download,” says Ed.
If you want to upscale the volume of data you are moving between remote devices and the Cloud, it can be done – using private network links such as AWS Direct Connect or Azure ExpressRoute. But these services don’t come cheap. “You can move data around in volumes but there are costs that would make AIVR prohibitively expensive,” says Barnaby.
So OBC had to find smart ways to reduce the volume of data flowing from its remote devices to the Cloud – ways that would preserve the high-quality information locked up in it.
One reason that AIVR generates so much data is the use of video. “Video is different,” says Barnaby. “Moving video around, storing video, and accessing and broadcasting video is difficult.”
Of course, One Big Circle was not the first tech company to wrestle with this problem.
Netflix delivers a smooth streaming experience – by using Edge devices to avoid delays in the Cloud. Photo by Unsplash.
Netflix and Edge computing
When video streaming service Netflix launched in the UK in 2012, it wanted a responsive service that would be instantly available to viewers in their own homes, without lengthy download delays. So in 2014 Netflix began installing Open Connect content delivery network (CDN) equipment in local BT exchanges all over the UK.
It required huge investment, but in return Netflix developed the capacity to store and deliver video availability from locations that were physically close to the consumer – successfully ridding the service of the latency that understandably irritates users: no one wants their Bridgerton session to glitch.
Netflix’s solutions were an example of the effective use of remote or local devices to process and store information at the edge of a network. It had found a way around the problems associated with distributing data at volume *from* the Cloud to consumers.
OBC’s problem, in contrast, was to capture data remotely and send it in the opposite direction – from devices at the *edge* of the network to the Cloud; but in both cases, *‘Edge computing’* offered solutions.
Edge devices operate on the remote boundaries of a network.
The Edge
The idea of ‘the Edge’ first emerged in the 1990s, and refers to the outer boundary of a network, where remote (also called ‘local’) devices interact with the Cloud. These are devices that only have one connection – to go into the network. There are no onward steps.
Edge computing brought data processing and storage physically closer to these outliers. The proximity reduced latency, improved efficiency, and enabled real-time responsiveness. “The Edge helped people do computing out in the real world,” says Ed.
It also helped avoid overloading the Cloud. If remote devices could be made more intelligent, *they* could do processing work, so the cloud didn’t have to.
If you can execute it correctly, Edge computing effectively turns remote devices into autonomous actors that interact intelligently with the Cloud, cooperating to deliver useful data, while withholding data without utility.
The Edge and the Internet of Things
The Edge helped to give life to the Internet of Things, another key component of the Fourth Industrial Revolution. The Internet of Things (IoT) refers to a network of physical objects, or “things,” embedded with sensors, software, and connectivity, allowing them to collect and exchange data over the internet, enabling automation and enhanced functionality.
Edge computing is growing. In 2021 only around 10 percent of data was not created and processed in a data centre or cloud; this was predicted to grow to 75 per cent by 2025, according to research firm Gartner.
A note on terminology
Now that we are familiar with Edge computing, it makes sense to stop talking about AIVR’s remote devices, or local devices, and call them Edge devices instead.
Problem-solving
Could OBC give AIVR’s Edge devices enough autonomy to figure out which data is needed – and send only this to the Cloud, while withholding useless data?
With so much data flowing into the Cloud, an ‘agent’ is required to control the flow – like a traffic cop.
AI in the Edge
While some models of Edge computing involve processing all data locally, or nearly all of it, that’s not practical for AIVR – there is simply too much data for its Edge devices to cope on their own.
Nevertheless, OBC engineers found that its Edge devices had sufficient processing power to apply a preliminary test of data quality – if they used artificial intelligence. This allowed the Edge device to set aside some low-quality data, easing the burden on the Cloud. How much? That depends on a number of variables, says OBC co-founder Ian Packer – more will be discarded in bad weather for example – but typically around 10 percent.
Which was progress, but to rid the system of latency, more was needed.
Dynamic Demand
OBC’s solution was Dynamic Demand, which characterises the relationship between its Edge devices and the Cloud, effectively automating the transmission and sharing of data, and ensuring that data flow is kept to a minimum.
There are two sides to Dynamic Demand; one manages the ability of remote devices to filter the data they share with the Cloud; the other, which we’ll come to shortly, manages demands in the opposite direction – the demands of the Cloud on Edge devices.
‘Behaviour’-driven data filtering at the Edge
OBC needed a way of configuring its Edge devices to automatically recognise certain kinds of data as more significant than others, and therefore more important to upload to the Cloud. So they called these types of data ‘behaviours’.
“‘Behaviour’ controls whether we keep recording, stop recording, or mark footage as a high-priority event, and upload it,” says Ed. “ ‘Behaviours’ are either geographic, or time-based, or event-based.”
An example of ‘event-behaviour’ might be the use of emergency brakes. This type of occurrence is clearly more potentially significant than a train pootling along at average speed – especially if it turns out to be part of a pattern – and therefore worth recording.
Using ‘behaviours’ to evaluate the usefulness of data, many of OBC’s monitoring devices now have restrictions on what they upload. These restrictions are configured differently, depending on the device and the route of the train they are mounted on, explains Ed. Some upload everything, some nothing.
For example, AIVR Focus – the line-scanning camera mounted beneath the train – uploads all the time. It photographs every millimetre of track, generating such a lot of data that the task of processing at the Edge would overwhelm the device. All this data must be processed in the Cloud.
AIVR devices regularly check some of the sleepier parts of the rail network, such as the East Looe Valley. Photo by Geof Sheppard.
Likewise, some more remote parts of the rail network are of necessity less frequently visited, so whatever happens, all the footage is uploaded, to ensure it’s as up to date as possible. For example, once a month, a device is fitted to a train on the 8 ¾-mile branch line that connects Liskeard and Looe in Cornwall via the leafy East Looe River valley.
In contrast, the AIVR devices mounted on the London to Edinburgh train, which makes the journey three times a day, do not of their own volition upload any footage unless there is an ‘event’ – an emergency stop, or a pantograph drop alarm. “We don’t want to swamp people with data,” says Ed. “Very prolific trains would drown out the more infrequent trains.”
The Cloud Agent demands the data it needs from Edge devices to complete a full picture of the rail network. Photo by Minerva Studio.
Cloud requirements: building the big picture
Meanwhile, the Cloud has its own information needs, and these are constantly changing.
That’s because only the AIVR data centre in the Cloud sees the big picture. After all, it’s here that all the data is assembled into comprehensive coverage of the network.
Think of the ‘big picture’ as a jigsaw puzzle. Using ‘behaviours’ to analyse the available data, Edge devices send what they consider the most important, and best quality, pieces of the puzzle to the Cloud.
The Cloud assembles those pieces, adding them to some pieces it already has, but finds that other pieces of the puzzle are still missing, or (to stretch the metaphor) require replacement.
For a real-life example, take the Rhondda line in South Wales. With a fresh round of data, automatically selected by the Edge devices on the 8.23 train to Cardiff and transmitted to the Cloud, the jigsawpuzzle is almost complete: there’s enough high-quality data to get almost complete coverage of the line. But unbeknownst to the Edge devices, the Cloud sees that the picture is still incomplete: the best available footage of a short stretch of line near the town of Tonypandy is out of date. New footage of that stretch is required.
So the other major function of dynamic demand is to give the Cloud the ability to tell remote devices what additional data it requires. For example, to request the most recently gathered footage of that stretch of track in Tonypandy.
The device must then be able to figure out whether the footage it has recorded of that stretch of line is of sufficient quality, ideally better than the older footage already on the Cloud. If it is, it will retrieve the data and send it to the Cloud.
Cloud agent
By what mechanism would this exchange actually be executed? The solution was to add a programme to manage the needs of the Cloud. As Ed says: “We introduced a new, automated decision-maker in the Cloud to look at all the footage available across multiple devices, and select footage for upload that would best improve the coverage and quality of the video available.”
The work of the Cloud Agent allows AIVR to continue to capture and transmit huge amounts of data from a wide range of deployed systems such as trains, but retains control over which data is uploaded and processed.
What are Agents, and why do they matter?
Leading figures in technology are super-excited about the use of agents or ‘agentic systems’. These seem set to be a huge step forward in automation.
Agentic systems are beginning to automate the execution of our wishes – a quantum leap for human technology. Photo by Pakin.
An agent is a software program or system that acts autonomously on behalf of a user, another program, or an organization. Agents are designed to perceive their environment, process information, and take actions to achieve specific goals without requiring continuous human intervention.
They can adapt and learn, and look likely to be able to use large language models such as ChatGPT to understand and execute natural language instructions. That means simply instructing your ‘agent’ in plain English (or whatever your chosen tongue) to do something for you, and the agent will figure out what you want and execute it for you.
According to Microsoft founder Bill Gates: “Agents are not only going to change how everyone interacts with computers. They’re also going to upend the software industry, bringing about the biggest revolution in computing since we went from typing commands to tapping on icons. Agents won’t simply make recommendations; they’ll help you act on them.”
“The Cloud Agent knows what coverage requirements are for different parts of the network, and combines both automatically uploaded footage with curated selections from available footage, to best meet the coverage and quality requirements across the network,” says Ed. “It looks for gaps and tries to find if there is a good fit somewhere out there on an Edge device.”
The Cloud Agent is not just retrieving data from devices, says Barn. It is growing in scope to other parts of the system, “Using artificial intelligence to apply another level of thinking to both manage data, and manage another level of processing.” He adds: “The Cloud Agent is an evolving system, trying continually to understand and feed back into the broader system.”
But how is the Cloud Agent learning?
Machine learning allows computer systems to improve their performance. Photo by NongEngEng.
Machine learning
Another of the key technologies that comprise the Fourth Industrial Revolution, machine learning is the branch of artificial intelligence (AI) that focuses on enabling computers and systems to learn from data and improve their performance on tasks without being explicitly programmed.
Machine learning thereby allows agents to adapt to situations – improving their ability to act autonomously, and boosting efficiency.
In the AIVR system, Edge devices use machine learning, and the Cloud Agent develops solutions and tests them. “The Edge device is learning all the time,” says Ed. “Cloud Agent uses this learning to make more intelligent decisions.” Barn adds: “Cloud agent decides what machine learning and what analytics is run on what data at what time. It orchestrates what processing needs doing and where – Edge or Cloud.”
Use Case 4: Thermal video
AIVR’s Forward-Facing Camera travels with a buddy in its housing: a thermal video camera. The images it generates reveal potentially hazardous hotspots on the tracks – hazards that may not be visible during a physical inspection. “When something is hot, something is not right,” says Jasper.
Using thermal camera images, AIVR identifies and logs hot spots on the rail – sure signs of potential trouble.
This data is especially helpful on electrified tracks, currently most common in southeastern England, where a ‘conductor’ rail bears a current of 750 volts DC. “Heat results when things aren’t performing properly – a track circuit bond is not attached properly, or some rubbish is rubbing up against it,” says Jasper.
Both can lead to bad things. The overheated circuit may trip out; the resulting loss of power can mislead the signalling system into thinking the track is occupied, causing red signals that delay trains. And if rubbish ignites, the fire can spread and may damage critical infrastructure, such as a signal cabinet – which can cause more costly delays.
AIVR uses machine learning technology to identify suspicious hotspots. It then sends automated alerts via email to key engineers in Network Rail’s Kent and Sussex route.
Once alerted, engineers can use the AIVR Platform to help make decisions remotely, referring to both the Forward Facing and Thermal Video, and to the hotspot’s history, to analyse the problem and assess risk levels.
Archive analysis
As AIVR draws on the vast and growing bank of data gathered, its authority has grown exponentially, too. That’s because the data archive is a resource that can be repeatedly tested and re-tested, allowing OBC to refine the algorithms they use to analyse the data. “Collect once, use multiple times,” as Ed puts it.
Conclusion
Near the beginning of the first part of this article, we discussed one of the most ancient and simplest of technologies: the quern, a pair of shaped stones with which, even before the rise of agriculture, our hunter-gatherer ancestors would grind foraged grain into flour. We finished by discussing some of the subtleties of 21st century computer technology.
Same old world, but we are interacting with it in ever more complex ways.
In the 300 years or so since the first Industrial Revolution, increasing automation has helped humanity to achieve its goals – at scale. The arrival of railways was one of the defining moments in this process, allowing people and goods to travel distances, more quickly and more cheaply.
As we saw in part one, the technology used to keep railways safe has also advanced steadily in the two centuries since the birth of the modern passenger railway.
Now technology is poised to make one of the most consequential advances in human history, with the continued unfolding of what is often called the Fourth Industrial Revolution (4IR). And this technology is proving highly consequential for railway safety, too. One Big Circle is harnessing some key elements of the 4IR to overcome historical challenges in rail inspection.
As we discussed earlier in this article, the railway is a complex system. “Analysing and improving complex systems can sometimes seem intimidating, even impossible,” says Barn. But fortunately, he adds, “There are ways of seeing the complexity, handling it, and even benefitting from it.”
The beauty of AIVR is that it provides a starting point, says Barn. “AIVR uses automation to provide a consistent flow of information. This helps rail engineers begin to cope with the complexity of the rail environment and to start solving problems as they occur.”
How did AIVR achieve this? Let’s recap. The first step in coping with complexity was to devise an automated system to observe it. To this end, OBC deployed an array of autonomous devices, capturing data from video cameras, thermal imaging devices, and other sensors. “We have had to do complicated things to see some of the complexity,” says Barn.
From the mass of data captured by these sensors, there follows a process of simplification. Engineers from a variety of disciplines discover patterns in the data and use this as the basis for further analysis and action.
In doing so, engineers are assisted by an increasingly sophisticated combination of artificial intelligence, machine learning and other advanced technology.
Building on automation: heuristics
Automation can only take you so far. Once you’re automated devices deliver the data you need, there remain important questions about the best ways to use it.
According to Barn, if the work of those sensors is to be used effectively, one thing is essential: mentality.
Essentially, engineers must set aside perfectionism in favour of building simple solutions that work. This is a process made feasible by heuristics – ‘simplified rules of thumb that make things simple and easy to implement’*.
Barn is talking about a comparative, layered approach that gets results in a complex environment – without requiring exhaustive analysis of every aspect of that environment.
And it is a continuously evolving process, Barn adds. Yet-to-be-discovered methodologies are set to further improve our understanding of the railway system. (These methodologies may themselves involve automation, machine learning, and other AI models.)
Thanks to this dynamic process, when exposed to stresses or challenges, railway systems will not only be more resilient, but also capable of learning and improving and thus becoming stronger – a state Barnaby calls ‘anti-fragile’ – from a phrase coined by the Lebanese-American writer and mathematician Nassim Nicholas Taleb.
And the heuristic approach is paying dividends. Automation through AIVR is already bringing tangible benefits to the industry and its workers. AIVR makes track inspection more efficient and reliable, and helps to reduce the time rail workers are exposed to potentially risky trackside conditions.
AIVR provides another example of the way automation can revolutionize traditional industries. It also reminds us of the potential for intelligent technology to reshape critical infrastructure for the better.
Meet the expert: Ed Torbett
One Big Circle’s automation expert Ed has been programming for 33 years, since the age of eight. He has a degree in computer science from Warwick University, becoming a full-time programmer from the age of 22.
Ed was just the seventh person to join the OBC team in 2019 (there are now more than 60). He created the original AIVR Go app, which originally ran on a mobile phone mounted on the train cab.
“I am an automation buff,” says Ed. “I actually have a lot of automation at home – loads. Like lights turning on when I walk into the room. My heating is based on how far away from the house I am. I made my own video doorbell.”