Understanding Visual Search in Mold Manufacturing

Software is a tool, like a screwdriver or a clothes dryer. It helps you get things done. But the invisible nature of how it works, and the remarkable number of problems it can solve in a short time, can make it seem almost miraculous at times. For example, consider a technology many have heard of but never use—visual search software that gives shops the ability to locate files using shape.

 

“Shaping” Your Search

First, a quick spin back in time. Personal computers began their unstoppable march across the workplace in the 1980s. As professionals grew accustomed to powering up their towers and monitors each morning, more and more data began to reside in a digital space instead of filing cabinets and blueprint drawers.

By the early ‘90s, people who, until recent times, had never used a computer suddenly had thousands of files in their folders. This new office practice created a new challenge: finding a file when you were not sure what it was called or where it was stored.

In the ‘90s, developers began work on desktop search tools, which became increasingly fast and accurate. Archie debuted in 1990, which was the first internet search engine. By 2006, the Oxford English Dictionary added the verb “Google.” However, as the use of text search grew, its limitations became apparent. The fundamental problem is this: The desired information is often defined by its shape, not by text.

Computer users, accustomed to split-second results from text searches, often found searches to be tedious—and ultimately fruitless—because of language constraints. They could not type a description that produced a match.

Another compounding factor was terminology. Three engineers working on nearly identical parts might find that one named the part a bracket, the other called it a flange and the third called it a base. And those are all people working in the same building and speaking the same language. Add in regional variations, plus 6,900 languages across the globe, and the text search tools proved frustrating.

In 2012, management consultants McKinsey & Company reported that “employees spend 1.8 hours every day—9.3 hours per week, on average—searching and gathering information. Put another way, businesses hire five employees, but only four show up to work. The fifth employee is off searching for answers, but not contributing any value.”

In a 2018 survey, workflow and content automation provider Nintex determined that 49% of professionals have trouble locating documents. So, this need to locate files by shape, plus the inability to do so, gave rise to the development of visual search technology for shapes stored in digital data.

Visual search for data is becoming as indispensable to industry as text search. Early adopters of visual search technology in manufacturing have already gained significant market advantages.

 

A Brief History of Recognizing Shapes

Although U.S. manufacturing has been very slow to adopt visual search technology, manufacturers in Asia and Europe are actively engaged in visual search, with measurable productivity increases.

Optical Character Recognition (OCR): Early efforts in shape recognition involved OCR, which identifies patterns and converts the letter patterns into some other medium. OCR has now been in use since 1914. In that year, two different OCR devices were simultaneously invented, both to aid the blind. In 1931, the “Statistical Machine” was developed, which could search microfilm archives using OCR. Microfilm had been in use since 1859, but until 1931, searches of microfilm were tedious and time-consuming.

Integrated Automated Fingerprint Identification System (IAFIS): Development of the IAFIS electronic system to replace the exhausting manual system of matching fingerprints began in the 1960s. By the mid-1980s, AFIS (later IAFIS) was in wide use by law enforcement.

Facial Recognition: The development of this law enforcement tool also began in the 1960s. Security forces use this in public settings, but the technology was first deployed at an airport quite recently, in 2011. Visual search for objects is sometimes referred to as “facial recognition for things.”

Shape search to aid the blind, to speed up archive research and to help with law enforcement each had a limited audience. Each generated interest, but had no application outside of its immediate circle.

Visual Search: The first attempts at developing visual search for products began in research settings in the 1990s. However, with the potential for broad commercial application, especially in e-commerce, visual search for products attracted many software developers’ attention. The first known introduction of this technology into the market was in 2002.

 

Breaking Down Visual Search

There are three types of visual search engines for shapes: Image to Image, 3D to 3D and Cross-Platform. These are designated in shorthand by replacing the word “to” with >, such as Image>Image.

1. Image>Image:

This type is used for product visual search, almost always retail. It works by using a photo of the desired product, such as shoes. It then uses this digital photo to find similar digital photos, either on the internet or on an online retailer's website.

As e-commerce continues to supplant brick-and-mortar stores, online retailers have discovered the benefits of allowing their customers to drag and drop a photo of the desired item or locate it with a phone app. For example, this option overcomes language and terminology barriers, since shoppers may not know the name of a product. It also allows online retailers to recommend similar products, since the consumer may be looking for products more or less like this one, not necessarily an exact match. Lastly, it greatly accelerates the shopping process by finding the product faster than a text search or catalog perusal.

For Image>Image web searches, the search engine produces results from its database of images scraped from millions of web pages. This is not proprietary data, but rather visual information readily available on the internet. A reverse version is technology that continually crawls the web, looking for photos that match those in its database. This is typically for finding unauthorized use of proprietary photos, artwork, etc.

For Image>Image searches in a private e-commerce database, consumers are encouraged to use visual search to locate a match, either with a mobile device or a desktop drag-and-drop. This search is limited to those items in that retailer’s e-commerce database.

In such instances, the visual search engine acts as a recommendation tool. It is not important to the online retailer that the consumer find an exact match. The website will bring up the closest catalog match and similar results.

Image>Image search requires digital photographs both as search input and as search targets. This works well. Consumer goods are commonly photographed from many angles for marketing, so there is an abundance of images available for search.

The technology then uses deep learning, a form of machine learning (or artificial intelligence), to become increasingly familiar with the data for more accurate results. That is, as users repeatedly search for a product using visual input, the Image>Image search engine makes a note of the correct match, as indicated by the shopper's selection, and uses that data to improve future searches.

2. 3D>3D

This category encompasses both two- and three-dimensional engineering drawings with searches conducted in a company's secure database, not on the web. A software system may specialize in one of these platforms, or both, or may cross between 2D and 3D. The designation 3D>3D is used here to include both.

Engineers use 3D>3D searches to find needed models and to avoid duplicated effort. This technology uses a complete or partial 3D rendering to locate existing drawings of a part and may first require text to narrow the category search.

Unlike Image>Image searches, which are useful for both finding and recommending consumer goods, the goal of 3D>3D searches is an exact match. The search returns may be ranked in order of probability of a match, but in an engineering environment, recommendations within a product category serve no purpose.

Using 3D>3D to find 3D models can overcome the following:

• Language barriers for companies that have engineers and designers creating models in different languages;
• Terminology barriers, since engineers, may use different nomenclature for the same part; and,
• Difficulties in naming protocols following a merger or buyout when engineers are accessing an unfamiliar database.

This search type also enables engineers to start a drawing, then pull up matches using the partial drawing to “save as” and work from existing models, eliminating duplicated work.

A further benefit is the ability to take an engineering model and search for duplicates. Large manufacturers frequently have parts that are so similar that they could be considered duplicates. But since they were developed independently, they require separate tooling, material specifications, etc., and are tracked with separate SKUs.

This is a significant financial drain for the manufacturer, yet it is impossible to know these duplicate parts exist without a visual search. This technology creates the possibility of consolidating the number of parts in production and streamlining inventory.

Cross-platform search technology allows users to perform the Image>Image and 3D>3D searches, as well as searches across file or platform types. This is significant for manufacturers.

 

3. Cross-Platform

Cross-platform search technology allows users to perform the Image>Image and 3D>3D searches above, as well as perform searches across file or platform types, such as Image>2D, 2D>2D, PDF>3D, Hand-Sketch>3D, etc.

This is significant for manufacturers. The critical issue for aftermarket sales is that millions of industrial parts are in the field that have never been photographed. With no digital photo in the database, Image>Image search does not work. Cross-platform search solves this aftermarket difficulty by using digital engineering drawings and PDFs as search targets.

Advantages of cross-platform searches include all Image>Image and 3D>3D search capabilities noted above, since this technology encompasses both of these technologies. Plus, it offers the ability to locate one file type using another, such as locating a 3D model with a photo or a 2D drawing with a PDF.

A customer in need of a manufactured component could even draw a hand sketch of the desired part or take a photo of a similar part and send it to the manufacturer. The quoting engineer then uses this as input to instantly pull up existing photos, PDFs, 2D and 3D drawings that match this geometry, dramatically reducing the time needed for quotation.

 

Summary

Visual search for data is becoming as indispensable to industry as text search. Early adopters of visual search technology in manufacturing have already gained significant market advantages. These are found in increased efficiency in design and engineering, parts sourcing, consolidation of duplicate parts and savings on unnecessary tooling. Those using cross-platform searches will also have the advantage in online quoting and aftermarket parts sales, quickly identifying existing parts based on shape. All of this leads to increased profit margins.