class GEM - C++ interface

SYNOPSIS

class LocalGEM, class RemoteGEM, declared in "gem/gem.h"

DESCRIPTION

The main function of GEM is to support the interpretation of sensor data on intelligent mobile robots by predicting sensor specific features (currently lines and aggregations of lines in the case of a video camera) that should be visible from a certain sensor position. Predictions can be taylored to various perception tasks like navigation, object recognition and state identification by several parameters including symbolic information like classnames and object states (see ftp://ftp.lpr.ei.tum.de/pub/papers/rovi/accv95-ah+nos.ps.gz for an introduction).

gemAccessContext * GEM::NewAccessContext(const nCamera &cam): To conveniently handle all necessary parameters each client has to allocate one or more so called "AccessContexts". All accesses to the model are issued by calling methods of gemAccessContext ; the parameter cam defines a pinhole camera model.

BASIC ACCESS

gemAccessContext & SetSensorPose(const nPose &p): Define the position and orientation (together: 6D-"pose") of the sensor (i.e. the videocamera, support of radar and laser sensors is currently only partially implemented).
gemSequencer<gemLine3D> GetLines(): Get the line-features that are visible from the current sensor pose.

GENERALIZED ACCESS

In addition to this most basic access several additional parameters can be set to support task specific predictions.

gemAccessContext :: MANIPULATING THE "FOCUS"

The representation of the environment in GEM has a tree-like structure. The "focus" can be regarded as virtual cursor, which selects the part of the tree that is relevant to the specific perception task. It also selects, which part of the model can currently be modified by the client. This part is always duplicated in the clients AccessContext to allow hypothetic changes and verification without interfering with other clients. The former focus on "world" (all objects and background) is in the new version replaced by the focus on an "island". An island has its own coordinate system, own objects and own background. The object classes are still global.

gemAccessContext & FocusIsland(CSTRING name): Objects and background belonging to the island name are taken into account for image prediction. The sensorpose refers to the islands coordinate system. This mode is intended for perception tasks doing localization or exploration/verification.
gemAccessContext & FocusWorld(): Focusses on the special island "WORLD". Objects instatiated outside of an island section are automatically assigned to "WORLD". For compatibility with old code and old data files.
gemAccessContext & FocusClass(CSTRING name): Only views of a private instance of the class name are generated. Modifications of this object don't influence the state of the model as it is seen by other clients. The sensorpose refers to the class coordinate system. This mode is indended for perception tasks doing object recognition.
gemAccessContext & FocusObject(CSTRING name): Only features of the object name are predicted but all other objects and the background of the island were the object is located are used for visibility calculation. The sensorpose refers to the islands coordinate system. Modifications of the object are performed on a private copy and don't influence the state of the model as it is seen by other clients. To make changes global, UpdateObject() has to be called. Intended for perception tasks which determine the state(s) of an object.

gemAccessContext :: MANIPULATING AN OBJECT/CLASS

When the focus points to an object or a class (i.e. a private object of a class which is instatiated in the origin of the coordinate system) the state of the object can be recursively set. Only one level of joints in the object tree is visible to the client at a time.

gemSequencer<char> GetJointNames(): Get the names of the joints in the currently focussed level. Immediatly after FocusClass() these are the joints directly connected to the fixed root-part of the object. The tree of joints can be traversed by shifting the "focus" up and down using FocusMember(jointname) and FocusUp() . After issuing FocusMember(jointname) the next level of joints, i.e. the joints connected to the member jointname comes into view.
gemAccessContext & SetJoint(CSTRING name, double state): Set the state of the joint name . Valid states range from 0 to 1, corresponding to the minimal and maximal angles or translations of a joint. In addition to that, three symbolic states, cut , unknown and preferred exist. If the state is cut no members of the subtree connected to this joint are taken into account, neither for visibility calculation nor for feature prediction. The state unknown results in the replacement of the subtree by its "mask", i.e. a worst case estimation for visibility calculation. The parameter preferred sets the joint to an a priori specified state, e.g. a door to the state 0 = "closed".
double GetJointState(CSTRING name): Get the state of the joint name .
gemAccessContext & SetAllJoints(double state): Set all joints below the current focus to state .
gemAccessContext & FocusMember(CSTRING jointname): Shift the focus down to the member connected by the joint jointname .
gemAccessContext & FocusUp(): Shift the focus up, that means toward the static rootpart of the object.
gemAccessContext & CreateObject(CSTRING name, const nPose &p): Only with focus on a class. Create one object of the currently focussed class at p and name it name .
gemAccessContext & UpdateObject(): Only with focus on an object. Transfer the state of the private copy of the focussed object to the "real one", i.e. make the state globally accessible.
gemAccessContext & SetObjectPose(const nPose &p): Only with focus on an object. Move the currently focussed object to p .
gemAccessContext & DeleteObject(): Only with focus on an object. Delete the currently focussed object.
const nPose & GetFocusPose(): Return the transformation between the currently focussed (member-) object and the world/island coordinate system.
CSTRING GetFocusIsland(): Get the name of the island where the currently focussed object is located.

gemAccessContext :: MANIPULATING THE "ZOOM"

The "zoom" is a second virtual cursor, which selects the features which are considered for the visibility test. Only features "below" the zoom in the object tree are used. The "zoom" normaly coincides with the focus and is set to it in each operation changing the focus.

gemAccessContext & ZoomObject(): Only with focus on object or class. Set the zoom on the root-part of the current object tree.
gemAccessContext & ZoomFocus(): Reset the zoom to the focus. This is done automatically every time the focus changes.
gemAccessContext & ZoomMember(CSTRING jointname): Only with focus on object or class. Move the zoom down the object tree, point on the member connected by the joint jointname .
gemAccessContext & ZoomFeatureAdd(long id): Zoom on a single feature (or a list of features, by calling this method repeatedly). Used after GetFeatures() to further confine the prediction.
gemAccessContext & ZoomUp(): Only with focus on object or class. Move the zoom up the object tree. When zoomed on individual features the the zoom points to the member afterwards, all features below this member become visible again.
gemSequencer<gemFeature> GetFeatures(CSTRING name): Prediction of general features: Get the ids of all visible features of the feature-class name . Define a set of interest by issuing ZoomFeaturAdd(id) . Following calls to GetLines() will only predict lines belonging to this set (Details in german: [zapf-da:rovi, kilincarslan-da:rovi]).

gemAccessContext :: QUERYING NAMES

gemSequencer<char> GetClassNames(): Get the names of known classes of geometric objects. A priori knowledge.
gemSequencer<char> GetIslandNames(): Get the names of known islands. Can vary during GEM operation as islands can be created and deleted.
gemSequencer<char> GetObjectNames(): Get the names of known objects. Can vary during GEM operation as objects can be created and deleted.
gemSequencer<char> GetFeatureClassNames(): Get the names of known classes of video features, at least "Line". A priori knowledge.

gemAccessContext :: MISCELLANEOUS

gemAccessContext & SetCoordinateMode(gemAccessContext::CoordinateMode mode): Predicted features are returned in world-, camera-, or chip-coordinates. Default is world.
gemAccessContext & SwitchVisCalculation(bool on): Switch the visibility calculation on or off (where possible, e.g. SetVoxel() and GetFaces() never support visibility calculation).
gemAccessContext & SetVoxel(nVector3D min, nVector3D max): Instead of predicting a sensor view access all features inside a certain volume. Used e.g. for initializing the DLM.
const gemBoundingBox & GetBoundingBox(): Get the bounding box of the currently focussed object.
gemAccessContext & CreateIsland(CSTRING name): Create a new island.
gemAccessContext & Dump(CSTRING name): Write the volatile contents of the model to a set of files. The optional parameter name specifies a unique fragment of the automatically generated filenames.

gemAccessContext :: ERROR HANDLING

In the case of a failure the AccessContext is set to a "high impedance" mode. It still accepts all method evocations but does nothing and returns dummy results. By this technique, the error status doesn't have to be checked after every call and passed upward through a hierarchy of function calls to the appropriate level of control. The "high impedance" mode cannot be left again, so the AccessContext has to be deleted and a new one has to be allocated. The error status can be queried at random by the following methods.


bool Failed(): Returns true when a failure has occurred an the AccessContext is already in its "high impedance" mode.
CSTRING GetFailureDescription(): Get as plain text description of the failure. For debugging purposes.
gemAccessContext & SetCatcher(Catcher *cp): Define a handler object that notified by GEM when an error occurs

gemAccessContext :: EXPERIMENTAL ACCESSES

Sequencer<gemFace3D> GetFaces(): Get all faces which intersect the viewing frustum. No visibility calculation. For first experiments with predictive displays.
gemAccessContext & StrobeZBuffer(): Generate a depth map for the current sensor view. Has to be called before GetZLine(), GetRegions(), GetFloxel() can be issued.
gemSequencer<double> GetZLine(int y): Get a horizontal line of the depth map.
gemSequencer<gemRegion> GetRegions(): Get image regions, i.e. the visible parts of faces as a list of spans.
const gemFloxel & GetFloxel(double x, double y, const nPose &delta): Get an optic flow vector at the normalized image coordinates x, y for 6D motion of the camera delta .
AccessContext & CreateFace(const gemFace3D &f): Create a background face. Currently the face is not incorporated into the s2d tree but stored separately which can impair performance when excessivly used. To be implemented correctly.

stoffler