I would say that ramming a floating object whilst sailing is the second most feared incidence for sailors, with a knocking out by seasickness in the first place. Nevertheless, most skippers would rather suffer from seasickness and vomit their guts out than collide with a lost container or a rogue buoy: Up until now, Radar was the only way to check for obstacles in a boat’s way when it comes to active detection of passive objects. Well, there is a new guy in town and his name is Oscar.
Talking to Patrick Haebig of BSB Group
This system is currently resounded throughout the land – at least here in Germany – since Boris Herrmann’s fateful collision with a fishing vessel some 90 miles off Vendeé Globe’s finishing line. I do not want to participate in the vivid cascade of speculations as to why the collision took place but rather found interest in the Oscar-system. In this I am happy that Patrick Haebig of the BSB Group agreed to talk to me about Oscar.
Lars Reisberg | NO FRILLS SAILING.com: “Patrick, first of all thanks for taking your time in these stressful days for talking to me. Let’s start with you as a person – where are you from, is there a personal connection to the sea and what did you bring to BSB to work on Oscar?”
Patrick Haebig | BSB Group: “Hi there Lars, thanks for having me on your magazine. Well, my position within the organization is simple: I am the responsible for Marketing and Growth in the BSB Group and as such I am in for making sure that people understand what Oscar is and how it works – and partly, also for selling this product of course. Talking of me personally, I am half French half Austrian with my father coming from Normandy, mum being from the Alps. Therefore I am closely connected to sailing and water sports since my earliest childhood days. Later I spent the summer regularly in France and also studied there, being outside on the water was always omnipresent for me, I even worked as a sailing trainer at some point in my spare time. So, yeah, the seas, boats and sailing, diving and surfing has always played a major role in my life.”
NO FRILLS SAILING.com: “How did you join the Oscar team?”
Patrick Haebig: “A friend of mine who used to be on the BSB-team from the beginning told me about the project and kind of invited me to meet Raphael, the inventor of Oscar and founder of the BSB Group. I was fascinated by the idea and the spirit and so I joined. It’s as simple as that. You see, the far more interesting story here is Raphael’s how Oscar came to life.”
NO FRILLS SAILING.com: “… don’t hesitate.”
Patrick Haebig: “Well, I know him since three years now and he very vividly recollects how Oscar’s idea was born. In a way this story connects to myself and as far as I am concerned to every sailor I know of as everybody have kind of the same experiences. Raphael back in the day had to bring his catamaran from France all the way down south to the Cape Verde Islands. It was in October and leaving France to the strait of Gibraltar was connected with very harsh conditions. Besides, it was cold, rainy and especially for the first couple of days at sea very dangerous and arduous. Off Spain, during another night shift, Raphael thought by himself: “Well, it’s pitch black, weather is shitty, it’s cold, I can’t and don’t want to be outside and even if I am there I do not see anything at all!” You know this by yourself, don’t you? You can see other ships – in good conditions – very good, mostly they do have active AIS. But even if you have Radar, you won’t see floating objects: Containers, trunks of trees, buoys cut loose or even wooden fishing boats … no chance. He thought: “You know what? We don’t see anything and it’s a thing of being lucky whether we hit something or we do not hit something.” After reaching the Cape Verde Islands, Raphael immediately started a research on the internet and found … nothing. This is when the idea of developing a collision detection device was born.”
NO FRILLS SAILING.com: ”So in a way it was his personal involvement as a skipper which set off the project.”
Patrick Haebig: “Yes, indeed. With BSB it’s not a basic requirement to be a keen sailor to work for the company, but it quite helps to understand and really know what sailing means and to know which hazards you have to deal with when underway. For me, it quite helps a lot understanding the situation of skippers, mostly family fathers having their loved ones aboard, being responsible for their lives. But also the budgetary situations of boat owners. And thinking about it, I can very easily relate to Raphael’s experiences: When I am sailing through the night I often have these thoughts as well. I try to brush them aside, as they are dark and almost always do not have a nice ending. But in a way it’s always the same conclusion: There is simply no way to detect floating objects ahead of the boat in the night. Think of a simple, funny MOB-practice: Even in good weather at day with just a little swell – the fender is very easily lost not far away from the boat. You just don’t see it anymore. It’s as simple as that: You cannot see floating objects when underway.”
NO FRILLS SAILING.com: “So, tell me please the idea behind Oscar and how it works?”
Patrick Haebig: “Well, next to AIS, which is mandatory on commercial vessels and will show up on one’s plotter to indicate position, speed and heading of another vessel, and Radar, which can actively detect objects around a yacht, Oscar is the missing link of both, as we like to call it. That is because each of the mentioned systems has its limits. AIS will only show vessels and some buoys – and this is only the case if they have activated the system. AIS won’t show any “dead” things like floating objects and it won’t show other sailing yachts, small power boats and such: Many pleasure crafts do not have a Class-B AIS transmitter. Other way round: Radar must be operated and monitored by the crew at all time. But Radar as well cannot show small objects and objects which are not reflecting the Radar-beams. So, there is a gap …”
NO FRILLS SAILING.com: “… and this gap is exactly the floating objects.”
Patrick Haebig: “Absolutely! Oscar is a visual system. It “sees” objects by scanning the forward area of the boat. It can see day and night as it features two thermal vision or infrared cameras, scanning for differences in temperature on the water. There is also one RGB camera which scans in visible light. The system is installed high up in the mast with forward view. It is constantly scanning the forward sector, looking for anomalies breaking the “white noise” of the wave patterns. Oscar’s brain as to call it is real artificial intelligence – this AI ten decides if this anomaly is a hazard or not.”
NO FRILLS SAILING.com: “So, how does it work?”
Patrick Haebig: “To understand how Oscar works let me tell you how we trained the system. In the first step we went out at sea and showed Oscar water. That sounds a bit silly at first, but that’s how it worked: We trained the system at first to learn what the medium water is like. It’s very hard in fact to deal with it: Just look at how many different sea states you can think of. Flat, calm water, waves, breaking waves with spray in a storm, crossing seas, dark and light water colors – we had to train the system the wide palette of different tastes in which the ocean comes. All these different sea states are “normal” water for Oscar. Quite a huge project to just get a default setting. Now, the art is when we train Oscar to detect anomalies. We call it an annotation. You are right: It’s a breaking of the sea state pattern, something that is not behaving like the current state of the seas. Oscar can detect that.”
NO FRILLS SAILING.com: “The system does that by checking for patterns in different thermal properties, right?”
Patrick Haebig: “Absolutely. The two IR cams can detect thermal differences of 0.06 degrees Celsius. Can you imagine? This is ridiculously sensitive. A warm head of a human being in the cold water is easier to detect – that’s 37 degrees to 4°. But even a buoy in the water that is just sprinkled with spray will show a difference in heat. Oscar should detect this.”
NO FRILLS SAILING.com: “So that sounds like “The Predator” or “The Terminator”?”
Patrick Haebig: “In a way you are right. The picture our system produces which is displayed on the screens is comparable from what you see in these movies. We aren’t working with bright colors, it’s more a gradation of white to black with white representing “warm” and black “cold”. But yeah, in a way this looks like “Predator” in the movie. But honestly, we have feedback from a skipper who told us the system showed a big algae carpet dead ahead – this is how sensitive the heat detection works. It’s an amazing technology!”
NO FRILLS SAILING.com: “Now, tell me a bit more about the artificial intelligence of Oscar. Is it a ready-made system you bought on the market or did you develop this on your own?”
Patrick Haebig: “We do have a team of some 15 dedicated software specialists currently working on Oscar. Not all are into the AI-project as some are working on the app, database and other projects, but we take pride in the fact that our AI is self-developed and tailored to the marine environment. Imagine this AI as a white paper or a newborn baby. It is not able to “do” anything as it first was a blank database with a program. As I mentioned before, we first trained the AI to be able to recognize the normal state. Once this was done, we began feeding pictures of anomalies and trained the system to classify these. So that’s hundreds of thousands of pictures of buoys, ships and boats of all sorts, floating objects and many many more. All of which in many states, different angles, sizes and constellations. You cannot buy this on the market. There are of course huge databases for example if you start something in facial recognition. You buy a package of a million faces and can feed it to the AI to train it what a face is. But that’s not the case with ships, buoys and UFOs. Same with the algorithm: The whole process of hazard recognition has been programmed and trained by ourselves.”
NO FRILLS SAILING.com: “This sounds like a huge undertaking.”
Patrick Haebig: “Oh you bet! I would say at first it sounds simple. But the more you look into it, it becomes more complicated. Right now, I believe, our set of pictures in the database comprises 55 million files. And now, with Vendeé Globe sailors coming in, all of which are set up with an Oscar in their masts, I would say with the next update we will hav .
NO FRILLS SAILING.com: “This is interesting: The artificial intelligence within Oscar constantly learns when underway?”
Patrick Haebig: “Well, in a way yes and no. When within Wifi-range, the user can update the system and put on the latest version of Oscar . Normally this is done via our smartphone app. When underway, Oscar is just working: Trying to detect hazards and put on a warning. A skipper can then decide to kind of “record” what Oscar does and make these data available to us. This is what we have done with current Vendeé Globe skippers who have huge data storage devices aboard to record their voyages. Remember: Oscar is a visual system, so the system is working with pictures – these are huge amounts of data indeed! Looking to the future, maybe if 5G technology or projects like Elon Musk’s Starlink will make transferring huge data amounts worthwhile, but as for now, we do not have a system in place that regularly or automatically transfers data back to us.”
NO FRILLS SAILING.com: “But there is exchange of data gathered by sailors?”
Patrick Haebig: “Yes, of course. Like with Vendeé Globe currently, we either ask skippers of whom we know that they are doing trips which may produce valuable new data or we are approached by them. In either case, it means installation of surplus storage capacities on the boats.”
NO FRILLS SAILING.com: “Let’s talk a bit about Boris Herrmann’s collision. There are numerous speculations going on and first statements or interviews have been released. I don’t want to dive into this, so let’s focus on Oscar: First of all – does the system have access to the steering or autopilot-system which could have prevented a collision?”
Patrick Haebig: “Yes and no. First of all it is absolutely not a real problem from a technical standpoint to have such a function installed. And we would have this ready soon. But we are not rolling it out immediately for two simple reasons: First of all we can’t take away the skipper’s responsibilities. If you rely too much on a technical solution, there may be individual people who completely forget to double check or even act. Secondly, think of people who are simply overwhelmed by a situation, in a busy area. They black out, crash – and afterwards we face a flood of complaints. It’s simply a matter of liability as well: If a system is fully automated it has to work 100 percent fail safe. And we all know: Nothing is fail safe. That’s why a Radar or AIS is as well not connected with the steering or autopilot-system: Like these, Oscar is producing warning signals. The assessment of the situation and the reaction to it has to come from the skipper.”
NO FRILLS SAILING.com: „Back to Boris Herrmann …”
Patrick Haebig: “Yes, of course. Well, I do not have any more information as of now. I have read the Yacht Magazine article as well as the interview with the skipper of the fishing boat. There are contradictory statements out there and as far as I am concerned and our team, we will wait until we get access to the very raw data. I can tell you that Boris was very positive about Oscar when we had contact with him. I must say that I am convinced that Boris tells us no lies or untrue facts: We have all seen him as a very steadfast, honest and straightforward man. He was always very open in every aspect, be it positive or negative. He always communicated very honestly about things that happened to him, be it his personal mood, a fissure in the sails or other problems. I simply don’t think that he is a man to obscure this incident.
NO FRILLS SAILING.com: “So, how reliable is Oscar working?”
Patrick Haebig: “Let me put it this way: One year ago the system was very prone to false alerts. We tackled it with high priority and so we worked together with a lot of beta-testers sailing all around the globe to train the system to be more reliable. For example, people like Francois Gabart on his Trimaran MACIF was one of our great partners since day one, Jean LeCam, Vincent Rioux or Armel Le Cléac’h joined the beta-fleet and collected huge amounts of very valuable data and experience. Even Raphael, the founder of BSB Marine, did test it himself: Last summer he jumped overboard and we checked if Oscar would detect him. Which it did. So, in the early days the system might produce one or two false alerts per hour, now we are down to one or two a day or less, which is quite good. As I said, the system gets better and better with every update.”
NO FRILLS SAILING.com: „How does the hazard detection work in detail?”
Patrick Haebig: „Well, it detects a break in the sea pattern by checking for differences in thermal pattern. Now, that sounds easy, but waves make it complicated. Even small waves may hide a floating object in the wave valley – so we had to train Oscar to check if an annotation – the anomaly – re-appears at roughly the same location. If such a detection is repeatedly appearing an alarm will be set off. That’s roughly one second – this is how long it takes until Oscar “decides” whether to “scream” or not. As a user you will really see the hazard: On the display the object will be framed like in crosshairs and – if possible – the object will be classified as a vessel, a buoy or simply an “unknown object”. Now the skipper can watch a live picture, and decide if this is a real danger to him and if a change of course is needed.”
NO FRILLS SAILING.com: „Where are the limits of Oscar?”
Patrick Haebig: „It’s a visual system so everything that is blocking your normal vision in real life is also hindering Oscar. Rain and heavy fog is the worst: The thicker the rain the less Oscar can see through it and detect thermal differences. Spray, light rain and even light fog is not such a big problem. Also the sea state is not of too big importance as we have quite good results right now. Oscar works best when thermal differences between the clutter and the object are big. But in essence, the detection range of the thermal IR cameras are 0.06 degrees Celsius, so in normal circumstances it’s quite reliable. At the day of Boris arrival there was quite some downpour – it may explain why Oscar may not have been working well. But we have to wait for the data, the rest would be speculation..
NO FRILLS SAILING.com: „Are you guys part of the official investigation?”
Patrick Haebig: „No, we are not. Of course we are ready to work with the team. We will nevertheless get access to the boat hopefully soon and then be able to retract data from the system – maybe we will have a clear explanation then. For us the most important thing now is to get an objective and straight answer to the question for our team: Have we done something wrong? Was there a fault in our system or a glitch from which we can learn? You see, I can change the wiper – but I cannot make the rain go away.”
NO FRILLS SAILING.com: „There are four versions of Oscar currently up for sale. What are the differences here?”
Patrick Haebig: „That is simple. We offer a simple, quick to install and ready-to-use version for I’d say the normal cruising and long haul sailor. That is the Oscar one 320. We have two thermal cameras and one RGB-cam. Everything is inside the Oscar-unit that is installed high up in the mast. There is one wire going down – the rest is displayed on the plotter or a mobile device. The cameras look forward in a 51 degrees angle for the IR and 110 degrees for RGB. Detection-AI is only using the pictures delivered by the thermal cams, the RGB is just for the skipper to enhance visual quality. It’s like the barrel man from the Golden Age of Sail up in the mast top. Our Advanced series is – like the name promises – more advanced: We have a camera resolution of up to 640 pixels, which is top-notch. The unit itself can rotate, for example to “look” into an anchor bay at night by steaming along without having to point the point into the bay. Or when you have a rotating mast, like on the IMOCA-boats. On this Oscar, thermal and RGB-vision is both used for detection. As we need more processing power, we have a separate unit installed in the boat which makes the Oscar lighter. See, the Oscar weighs 750 grams. For an IMOCA, having 750 grams in the mast means approximately eight (!) kilogram more ballast in the keel. That’s quite a factor. We as well offer a custom solutions which can be from … to… if a client has special requirements and we are able to develop it.”
NO FRILLS SAILING.com: „What is the price for the Oscar system and where can a skipper get it?”
Patrick Haebig: „We are offering the Oscar one 320 for just under 10.000 Euros. Racing skippers or people going for the advanced versions would be needing a budget of 24.000 to 35.000 Euros for their Oscars. But these are now mainly used by bigger, faster boats, professional one- or shorthanded racing crews with bigger budgets. For the ordinary family skipper sailing the seas the Oscar one 320 should as well be an attractive choice. We have set up a network of retailers and boat dealers who are actively offering our product, people could also check our website or contact us directly. We are also talking to yards. I cannot disclose as of now which brands are currently discussing including our Oscar-system in their boat offers, but I can tell that for example German sailboat maker-legend Michael Schmidt who is the founder of Hanse Yachts was among the first commercial users to offer Oscar for his brand Y-Yachts and has already an Oscar installed on his demo boat Y7 CIN CIN. We will see what we can do to spread the message.”
NO FRILLS SAILING.com: “At the end, Patrick: What will the future bring? Where do you see potentials for improvement and maybe trends in safety at sea?”
Patrick Haebig: “When it comes to Oscar I can say that in general we are very happy with the current state of the product: We are utilizing state-of-the art equipment, for example, the camera technology with 640 pixels resolution is absolute top-notch, also the graphic processors and computing power we are using. So here we are at a peak. Speaking about the software, well, it’s inherent that you are never finished. It will never stop to look for better performance, minimizing glitches, upgrading algorithms. What is a big topic for us now is further integration of our system into existing systems. That is the user interface, our app. But also integration with the big brands providing centralized management solutions. Raymarine recently integrated Oscar into their service, which is a big step forward. Here’s currently the biggest undiscovered land for us. Speaking of the AI we are constantly working on making it better and better in recognizing and identifying objects, and thus lowering false alerts. In this, the system will work better and better, honoring the trust of our clients.”
Patrick, thanks so much for this deep insight – all the best for you guys and I hope we can meet in person soon one one of the upcoming boat shows.
Pictures with kind permission by BSB Group © Copyrights by E. Stichelbaut, A. Beaugé, J. Lecaudey, G. Lebec, L. Viboux and T. Deregnieaux
You may also like to read:
Collision avoidance gone wrong
Practicing the code of conduct in MOB
Survival suite test