Object Detection Technology: The Next Big Thing
Object detection helps automate tedious tasks, increase production environment safety, and find everything from manufacturing defects to a parking spot. It has a ways to go for more nuanced tasks, but its prospects are bright.
Seagate Technology inspects tens of millions of computer storage drive parts daily. The media that goes into them. The heads. The wafers. The completed assemblies and outer shells.
For much of the company’s history, this inspection was a time-consuming and costly process. To achieve the quality the global company’s customers expect, hard disk and solid-state drives would be tested once, twice, or even three times before human beings gave them a final blessing in what’s known as sequential inspections, which involves looking intently at parts through electron microscopes.
But with the rise of AI-enabled object detection technology, Seagate is automating inefficiency out of its production processes and bringing in high-speed consistency. To achieve this increased throughput, the company is now installing high-powered edge servers closer to the point of production to run its machine learning models and expand its Fourth Industrial Revolution technology to assembly lines in factories around the world.
AI technologies at the edge
“Our quality systems use a multitude of different types of measurements that are image-based,” says Bruce King, a data science technologist at Seagate. “It’s a rules-based system that is programmed by engineers. The opportunity that we have with artificial intelligence is that we can have the machine learn all the rules so we don’t have to program them. This saves the engineers quite a bit of time and effort. And we’re finding that the machine learning tools are very flexible. They can find all the same things that a rule-based system can. They self-learn, so to speak. And their accuracy is as good or better in all cases that we’ve analyzed so far.”
Seagate is one of hundreds—maybe thousands—of companies that are either planning, piloting, or deploying machine learning with deep neural networks, algorithms patterned after the human brain, to solve real-world business challenges. The good news is that technology techniques like deep learning, and a treasure trove of publicly available frameworks and libraries, have made advanced analytics commercially viable and able to scale. The technologies may vary, and opinions may differ, but like any emerging technologies and standards, arriving at a clear consensus regarding platforms and approaches will continue to evolve. In the meantime, companies like Seagate are getting an early-mover advantage over their competition.
Some of the common debates are around what to sensor, which high-resolution camera to use, which aggregation and data transport software to use, and how to manage and secure the devices at the edge. Organizations must also consider what edge platform is best suited for a majority of their use cases, to minimize complexity and sprawl, as well as the need to hire, versus contract, super-smart data scientists to make object detection work, as demand for this scarce talent pool grows.
Whatever the case, there is clear momentum behind object detection technology. Manufacturers, local governments, transportation authorities, marinas, schools, hospitals, and other vertical industries are increasingly embracing projects built on it.
Factories of the future
In manufacturing, for example, Foxconn, a Taiwanese multinational electronics contract manufacturing company, is about to launch an object detection program to inspect every Hewlett Packard Enterprise server and component it produces for distribution in Europe. Indeed, its Kutná Hora plant in the Czech Republic is designed to inspect multiple products coming down the line and can also examine any variety of configurations rolling along, regardless of their order. The system will examine the objects, compare them with images in the company’s database, and with the help of video analytics and AI, determine whether anything looks amiss. In short, the system enables Foxconn to identify any potential failures in real time, which is critical given the company’s efforts to ship orders to customers like HPE as rapidly as possible, while ensuring consistent high quality across the products produced.
“The system compares what’s been assembled to what it believes should be assembled,” says John Gallagher, operations manager at Foxconn CZ. “It will look to see if everything is where it should be. Are the processors in the right spots? Are the cables in the right sockets? Are the drives correcting? And it will go through the whole configuration that we ask it to identify. The benefit of all of this to us is that we try to ship 50 percent of the orders we receive within 24 hours. This [object detection technology] allows us to be more predictable for customers. It allows us to have shorter manufacturing times for them as well. And it helps remove failures from the process.”
Gallagher says the company has high hopes for the pilot project, which kicked off in late March. If all goes well, Foxconn could expand the technology to other continents.
Smart transportation
Object detection technology is also showing great promise in the world of transportation, most notably with how people park their cars and moor their boats.
If you drive to work, you may dread the usual slog of trying to find parking on a city street or in a garage. In the near future, this might not be so difficult, thanks to smart parking solutions.
Instead of driving around for 10, 15, or even 30 minutes trying to find a parking space, you whip out a mobile app that locates a parking spot for you. On the back end, some combination of IoT parking meters, sensors, video cameras, radar, analytics software, DNN programs, edge servers, and mobile payment systems might come together to find that lone spot at the corner of 5th and Market—and even reserve it for you. In fact, you might eventually be able to sign up for an AI-driven service that knows your typical driving habits—when you go to work, when you stop off at the neighborhood bar, and when you go home—and prearranges everything before you even think to do it yourself.
Sure, that’s all pretty futuristic. But the technology is evolving so quickly that it’s not beyond the realm of possibility. And we are already seeing startups and local governments as well as transportation authorities working together to try and make something happen before too long.
In fact, one research firm says the smart parking technologies market is expected to eclipse $5.51 billion by 2024, with a compound annual growth rate of 19 percent.
“The provision and management of parking within cities is critical for the city’s ability to function and thrive,” notes Mark Zannoni, IDC research director for smart cities and transportation. “Parking impacts the urban economy and quality of life, as it provides access to businesses and events, affects traffic, and provides sizable revenues to local governments. Accordingly, more and more cities worldwide are looking at smart parking as a service offering while they transform themselves into smart cities.”
Los Angeles could be considered one such smart city. Since 2012, its LA Express Park program has sought to make more parking available in a 4.5-square-mile downtown area by giving commuters several ways to find parking spots. This has always been a huge issue for the City of Angels, where much of downtown peak traffic congestion is thought to be caused by drivers hunting for parking.
Through LA Express and work being done with a San Francisco Bay Area startup called Streetline, the city has slowly expanded the use of smart parking meters, parking space vehicle sensors, real-time parking guidance systems, and an integrated parking management system to other districts. Streetline has partnered with other California cities, including San Carlos and San Mateo, to save people nearly 713,000 hours of driving, 3 million miles, and 178,00 gallons of gas, according to its website.
“We are still in the position that hard, real data is required to power real-time availability of parking information,” says Mark Noworolski, CTO at Streetline. “Connected cameras with an object detection back end are a natural fit for that, as they are less invasive than on-street sensors and can be deployed in a wide variety of environments.”
The same technology that may save you time on the road could also help you accurately anchor your boat. At the recent Dusseldorf Boat Show in Germany, FLIR Systems, a Wilsonville, Oregon-based imaging and surveillance systems company, announced what may be the marine industry’s first intelligent object recognition and motion sensing “assisted docking solution.” Called Raymarine DocSense, the company’s vision camera technology and video analytics integrate intelligence gathered from surrounding imagery with the vessel’s propulsion and steering system to assist boat owners in tight-quarter docking maneuvering.
Campus security
One of the more profound and meaningful applications of object detection technology will be how it helps protect children.
Since the Columbine school massacre in 1999, more than 187,000 students have been exposed to gun violence, according to a Washington Post study. It’s a tragedy that continues to grow and intensify with few good solutions. For example, some suggest deploying armed guards while others advocate hoisting security gates on campuses. Neither approach has much support.
But what if you could almost invisibly identify whether guns, knives, or other weapons—concealed or visible—are coming into schools? If you could do so without violating privacy rights, wouldn’t it be interesting? And wouldn’t it be worth discussing possibly expanding such technology to other threatened areas, such as airports, bus stations, hotels, casinos, movie theaters and shopping malls?
Numerous startups not only believe it’s possible but that they have a calling to do so.
“This is a relatively new field. But given the urgency of the [mass shooting] problem, there are a number of players looking at how to address the challenge,” says Martin Cronin, CEO of Canada’s Patriot One Technologies, which offers a concealed weapons detection system built around a cognitive microwave radar device and custom AI software. “There is no question that embedded, discreet sensors, powered by AI, will become more ubiquitous for public safety.”
Cronin acknowledges that any discussion about monitoring people to see what they’re carrying—even if it’s guns—is always controversial. For that reason, he says it is imperative to involve the public in all decisions around security measures.
“There are understandably public fears about slipping quietly into a mass surveillance society,” Cronin says. “Our technologies focus on the threat objects (guns, knives, bombs, etc.). The person is of no interest to the system unless they are carrying a threat. That is important to maintaining public confidence and for assuring that there is no discrimination involved. We also generate no body image in the process of scanning for weapons with our radar-based systems, which is important for privacy. This is an approach which we feel can maintain the important public consensus. People want to feel safe, but there is a limit to what they are prepared to give up in their civil liberties to enjoy that safety.”
Careful advancement
Object recognition technology is in its early stages. While it has begun to transform the manufacturing floor because the system works with known objects, the long-term vision for the technology is to infer things about objects and learn from those experiences. AI and machine learning techniques using multilayered applications of deep neural networks will help it to eventually be able to pick out almost any object in a crowd. But it’s not quite there yet. A team of cognitive psychologists from the University of California, Los Angeles, challenged some of the best deep learning networks to look at color images of animals and objects, with the images altered. For example, the surface of a golf ball was displayed on a teapot; zebra stripes were placed on a camel; and the pattern of a blue and red argyle sock was shown on an elephant. The learning networks ranked their top choices and chose the correct item as the first choice for only five of 40 objects.
“Deep learning systems process information very differently from humans,” notes Philip Kellman, a UCLA distinguished professor of psychology and senior author of the study. “The biggest misunderstanding is that AI devices are ‘seeing’ the way we are. That’s not a comment about having consciousness. When a deep network is shown a picture and says ‘horse,’ it does not have a description of what the shape of the object is [and] does not perceive that it is detachable from the surfaces behind it. Conversely, if you view a certain glass ornament on my desk, you will immediately see that it is a goose, because you see its shape. Deep learning systems aren’t seeing that shape at all. And shape is not the only abstract property that’s being missed—but it is among the most crucial. In human vision and learning, the fascinating fact is that you don’t need to exhaustively show all possible views and versions of the objects that you want the network to learn because when we acquire abstract descriptions, like shape, we have a more powerful way of generalizing.”
