Connectivity Solutions Will Not Help with Autonomous Farming

Over the last two years, agricultural OEMs have been in a rush to get data from their machines to their cloud servers. Claas uses Proemion’s connectivity solution. Agco, Grimme, Krone and a few others bet on AgriRouter running on SAP’s cloud servers. Holmer seems to favour Bosch’s IoT suite. All these solutions have one thing in common: They transfer little data, typically less than 50 data points per second. They top out at 100 data points. They are good for task or order management, asset tracking, fleet management, monitoring of machine health and remote diagnosis.

At least some of these companies seem to think that these connectivity solutions will help them with driver-less or autonomous farming like harvesting, seeding, spraying or fertilising. The wishful thinking would go like this.

The farm machines transfer data to the cloud. The data is used to train machine learning algorithms on extremely powerful compute servers. The model, the result of the learning process, automagically detects whether grains are dirty, maize leaves are dry or the heads of sugar beets are chopped off. Typically, this model runs in the cloud as well, because it requires more compute power than is available on the farm machine or because it adapts to new conditions and self-optimises by continuous learning. This is the way how voice assistants like Alexa and Siri understand spoken language or how medical software recognises cancer cells in MRT scans.

This approach does not work for farm machines. Here is why and what we can do about it.
Continue reading

Agritechnica 2017: What’s New for Terminals?

Agritechnica 2017 was my third visit after 2013 and 2015. My focus was on terminals (display computers) as usual.

The standard terminal of 2017 is powered by a quad-core NXP i.MX6 processor (32-bit ARM Cortex-A9 with ARMv7a architecture) and has an HD 12-inch multi-touch display (resolution: 1280×800, format: 16:10). The new ISOBUS terminal CCI 1200 manufactured by CrossControl is the prime example.

In 2013, there were only terminals with single-core Cortex-A8 processors (NXP i.MX53). In 2015, there was only the odd prototype terminal with a quad-core i.MX6 in 2015 (from Grammer Belgium) but no production-quality ones. In 2017, most terminals sport a quad-core i.MX6 (Cortex-A9) processor. The processing power of terminals increases very, very slowly.

Compare this to a typical processor used in today’s in-vehicle infotainment systems. For example, the Renesas R-Car M3 sports two Cortex-A57 and four Cortex-A53 cores (all 64-bit), which has the performance of low-end to mid-range desktop PCs. Agricultural terminals need this procesing power as well, if the agricultural industry is serious about autonomous seeding, spraying and harvesting.

CLAAS demonstrated a first step into this direction. A camera is trained on the crop flow. An image recognition software (most likely using machine learning) detects whether the grains are too dirty and whether there are foreign particles between the grains. The future will see more and more such software to deduce actions from sensor data.

These expensive computations must be performed onboard the machine, because the Internet connection is not good enough on the field to send the data to powerful servers and to perform the computations there. The most powerful computer on the machine is typically the terminal. Current terminals are not powerful enough for these computations.

One terminal stands out from the uniform, slightly boring bulk of 12-inch, 1280×800 and quad-core i.MX6 terminals: the PowerView 1200 from Murphy by Enovation Controls. It is powered by a dual-core Cortex-A15 and has a 12.3-inch multi-touch display with a resolution of 1280×480 (format: 8:3). The PowerView 1200 is well-suited for dashboards and can double up as a rearview mirror.

I’ll take a more detailed look at some terminals in the rest of this post. I will ignore quite a few terminals, because they don’t stand out in any way from the rest or I simply overlooked them.
Continue reading

CAN Standard Helps Attackers to Switch Off Brakes, Steering and Engine

Security researchers from TrendMicro and the Technical University of Milano explain in a blog post “The Crisis of Connected Cars: When Vulnerabilities Affect the CAN Standard” and a video how to switch off any electronic control unit (ECU) attached to the CAN bus. An attacker could switch off the brakes, engine or steering of your car. The researchers exploit a feature defined in the CAN standard. Every vehicle using CAN bus – so, basically every car, truck, harvester, tractor and construction machine – is open to the attack.

Let us assume that the attacker wants to switch off the brakes. If the attacker has local access to the vehicle, he only needs to attach a CAN device with a malicious version of the CAN stack to the CAN bus. It is as simple as connecting the two wires of the CAN device to the respective wires of the CAN bus anywhere in the vehicle. Such a CAN device costs not more than 15 Euros and the CAN stack software is freely available.

If the attacker only has remote access, he must hack into the infotainment system, the terminal or the telematics box of the vehicle. The famous jeep hack shows how to do this. It is much simpler for agricultural and construction vehicles than for cars, because security is often neglected. Once hacked, the CAN driver is replaced with a malicious version.

The malicious version of the CAN stack abuses a feature of the CAN standard to deal with bus contention. Whenever the ECU of the brake writes a frame to the CAN bus, the malicious CAN stack immediately sends a frame where a bit of the original frame is flipped. The brake recognises a bus contention and sends a highest-priority error frame to recall its original frame. This error frame tells the other bus participants to ignore the original frame. After the malicious stack has killed 32 frames from the brake in this way, the CAN bus takes the brake ECU from the bus. The brake is switched off.

A full fix for this security flaw would require a change of the CAN standard. This will take some time, probably a couple of years. There is no easy way to mitigate the risk of a local attack. This is especially a problem for tractors, harvesters and construction machines, which can easily be accessed locally. You can make a remote attack harder by signing the CAN stack cryptographically.

Farmpilot Could Have a Bright Future

Farmpilot is a software to help farmers and contract farmers with their daily logistic tasks. It helps with planning at which location drivers and vehicles have to be at a certain time. It also helps tracking the fuel consumption, distance and area of the vehicle on the field and road – ordered by field and by customer. Contract farmers use this information for invoicing the landowners and farmers for paying their employees.

Many farmers use farmpilot already. Two of the biggest sugar producers in Europe, Nordzucker and Südzucker, have mandated the use of farmpilot for harvesting sugar beets (see page 3 of this PDF). Of course, farmpilot sends its data to the cloud. And – Arvato, the company developing farmpilot, has the cloud infrastructure in place.

Probably every terminal in a tractor or harvester comes with a task or job management. The manufacturers of these terminals should replace their home-grown solutions by farmpilot and save the development effort. There are three ways how to do that.

First, the farmpilot application runs on a tablet and communicates with the terminal over (W)LAN. The tablet has a mobile Internet connection and synchronises its data with the farmpilot servers in the cloud. The terminal relays data like fuel consumption, distance, harvested area and GPS coordinates from the CAN busses to the farmpilot application on the tablet. The driver interacts with the application solely on the tablet. This is most likely what we will see soon in the sugar industry.

Second, the application is mirrored to the display of the machine’s terminal via VNC – similar to MirrorLink, Apple CarPlay and Android Auto for in-vehicle infotainment systems. The application still runs on the tablet or smartphone, but the user interacts through the terminal’s touch screen with it. This is more convenient for the driver, because the terminal is usually closer to the joystick and the steering wheel than the tablet.

Third, the farmpilot app runs on the terminal directly and communicates directly with the cloud servers. There is no need for an extra tablet or smartphone anymore. The app could use something similar to the AWS IoT Device SDK, which sends MQTT messages to the cloud and receives MQTT messages from the cloud in a secure way. Using a cross-platform GUI and application framework like Qt, Arvato could develop their app once and deploy it on all terminals, tablets, smartphones and PCs. This would be the best solution for the user and the solution with the least development effort.

This is not yet the bright future, but the necessary groundwork Arvato must do. In its current version, the farmpilot application relays a few CAN messages to the cloud. The infrastructure to forward every relevant CAN message from the engine, the harvesting header or the drive train to the cloud is in place. The farmpilot app is the gateway to all the data from the harvesters and tractors. Once this data is in the cloud, Arvato could run deep learning algorithms to find the right header or implement settings for an optimal yield or to detect machine defects early.

Now, this would be a bright future for farmpilot.

Grimme ErgoDrive: Probably Not That Ergonomic

Grimme moves the HMI for changing the nominal values of the 80 most important harvesting parameters from the driver terminal(s) to an extra piece of hardware – called ErgoDrive. Four rotary knobs next to a small display enable the driver to change the nominal values of the 80 most important harvesting parameters.

As the driver looks at the display at a flat angle, he will probably have difficulties to see anything on the display. He must either memorise which of the four knobs does what or bent forward. Is this really more ergonomic and easier to use than showing the direct-access parameters on the terminal screen and changing them through a special rotary knob?

Another thing is clear: Moving software functionality into dedicated hardware is certainly more expensive or less economic for the customer.

Revolutionary HMI-Design of Forage Harvester

Agricultural OEM Krone bets on Qt software library for development of their terminal / First prototype read in less than three months

Big X in maize harvest at day time

Big X 480 in maize harvest

585 horsepower, 15.6 litres engine displacement and an up to 9 metre wide cutterhead – forage harvesters are among the most powerful agricultural machines. It takes a lot more technology and know-how to drive such a vehicle than a car. The driver terminal of a forage harvester must process information from more than 30 components like motor, cutterhead, metal detector or grinder within tenths of a second. The terminal additionally provides a diagnosis system. The agricultural OEM Krone from Lower Saxony has built all these functions into the touch-screen terminal of its forage harvester Big X 480/580. The HMI software of the terminal was developed with the GUI and application library Qt.
Continue reading

HMI Trends from Agritechnica 2013

At this year’s Agritechnica, I had a close look at the HMIs of the tractors and self-propelled harvesters of CLAAS, AGCO (Fendt and Massey Ferguson), John Deere, New Holland and some others. My focus was on the UI software running on driver and auxiliary terminals. I think that these HMIs have a lot of catching up to do with respect to in-vehicle-infotainment systems and even more with modern smartphones. Here is my detailed report. Continue reading