A novel robotic-vision system that combines object pre-marking and active visual sensing was developed for 3D-object recognition. In this system, an object is modeled by a small number of its 2D perspective views each associated with a corresponding viewing axis. A mobile camera is used such that its optical axis can be aligned with one of the standard-viewing axes of the object. The matching process is thereafter performed between the acquired 2D standard-view and a library of standard views determined a priori for a set of objects.

Object pre-marking (via circular markers in our case) serves three purposes: specify the standard views, define local surface normals (i.e., standard viewing axes), and convey information on the 3D location of the object. Active vision, on the other hand, facilitates the acquisition of the standard-views of the objects. If the first view acquired at camera’s initial location is insufficient to identify the object, additional standard views are acquired at camera’s secondary locations.

Following the development of necessary analytical tools for the on-line recognition process in the earlier phases of our work, CAD-based off-line planning issues were addressed. The off-line-planning phase for the proposed vision system involves optimal pre-marking and optimal camera placement. The former is defined as the determination of the minimum number of markers, and their best locations on a given set of objects to yield maximum distinctiveness between the 2D standard-views. The latter targets the determination of the camera’s optimal initial and secondary locations for efficient marker detection. The two problems addressed above were first solved individually, and then solved in a combined manner, namely, obtaining simultaneously the optimal marker arrangements as well as the optimal camera locations.

Recently, as a natural extension of the (static) active object-recognition system (ACTOR), described above, a moving-object recognition system (MORE) that integrates many of the components of ACTOR was also developed. In MORE, a Kalman-Filter-based motion-estimation method is utilized for tracking the moving circular features. The predicted poses of the markers are subsequently used to guide a mobile camera to appropriate object-viewing positions.

One of the major difficulties in applying the circular-feature pose-estimation method to moving objects is the orientation-duality problem. To solve this problem, we formulated several methods, based on the analysis of consecutive images, by using constrained object motion and additional surface features. Ill-conditions which lead to the failure of the methods were also investigated analytically.

In regard to implementation, our experimental set-up comprised a six degree-of-freedom GMF-100 robot with a CCD camera attached to its end effector, an additional fixed camera to track the object motion, two PCs and proper scene illumination. All off-line-planning activities were carried out on a SUN workstation which ran the SDRC’s I-DEAS CAD package.