Team CajunBot – Source Selection Sensitive Technical Approach Team CajunBot Team CajunBot – Source Selection Sensitive Table of Contents 1. Introduction .......................................................................................................................................... 3 1.1. 2. 2.1. 2.2. 3. Base Software Architecture ............................................................................................ 4 Reliability and Fault Tolerance ....................................................................................... 5 General Approach ................................................................................................................................ 5 3.1. 3.2. 3.3. 4. 5. Current Capabilities ........................................................................................................ 3 Addressing the Urban Challenge (UC) ................................................................................................ 4 Enhancing the Sensor Configuration .............................................................................. 7 Encoding Situation-action Rules..................................................................................... 8 Sensor Fusion and Reasoning Under Uncertainty ........................................................ 11 Summary and Closing ........................................................................................................................ 12 References .......................................................................................................................................... 12 Team CajunBot – Source Selection Sensitive 1. Introduction Team CajunBot of the University of Louisiana at Lafayette proposes to develop an AGV to meet the requirements of the 2007 DARPA Urban Challenge (UC) event. Team CajunBot was a finalist in the last two DARPA Grand Challenge events. For the 2007 UC event, Team CajunBot will use a Jeep Rubicon Wrangler as its base vehicle. The Wrangler, dubbed Ragin’Bot (Fig 1), builds on the experiences and software developed from the CajunBot, but uses a more Fig 1. The Ragin’ Bot AGV in action. conventional platform. Ragin’Bot was acquired by the university over a year ago to strengthen its ongoing research in autonomous ground vehicles. Using proven software algorithms from the CajunBot program [Lakh], Ragin’Bot (RB) is already a fully functional autonomous robot. This proposal requests funds to improve the sensor configuration of RB, its associated hardware, and to further develop the software of RB to achieve the safety and performance objectives of the Urban Challenge (UC). It also seeks funds for testing and assessing the reliability of the vehicle and its subsystems during development. The following narrative summarizes our current AGV capability and explains the technical approach we shall use to meet the new challenges of the 2007 event. 1.1. Current Capabilities Ragin’Bot follows a sequence of waypoints while avoiding obstacles, including hay bales, traffic cones, tires, cars, and tank traps in the presence of transient GPS outages. It is outfitted with a cluster of computers in the back of the vehicle. The software implements a shared-memory, blackboard in middleware on a distributed memory platform. The current algorithm is sufficient for generating vehicle navigable paths in the free zones consisting of static obstacles, as needed for the 2007 UC, although it will have to be enhanced for moving obstacles. We have a mature technology for generating vehicle-navigable obstacle avoidance paths in a grid world. Fig 2. Current vision-based scene description abilities currently running at 5 Hz. Left: Original scene. Middle: Road coded in red and lane markings in white. Right: Horizontal green lines show calibrated distance markings. Team CajunBot – Source Selection Sensitive In addition to transferring the 2005 Grand Challenge capabilities from CajunBot to Ragin’Bot, we have added new capabilities to specifically address the UC, as follows: 1) Developed the ability to compute distance to lane boundaries and road boundaries using color camera, as demonstrated in Fig 2 [Felz]. 2) Developed primitive control action for following a vehicle at a safe fixed distance. 3) Developed the ability to detect curbs using lidar data. 4) Developed the ability to visualize the AGV system data in a 3D stereo visualization facility. The system can visualize any data flowing on the middleware, which includes raw sensor data and data flowing in the processing pipe. 2. Addressing the Urban Challenge (UC) The following describes our technical approach to enhancing the capabilities of our AGV to meet the UC objectives. 2.1. Base Software Architecture The team’s solution may be classified as a hierarchical context-based, 3–layer architecture [Gat] consisting of control, sequencing, and deliberative algorithms. DARPA’s UC requirements will be organized in a hierarchy of contexts. These contexts describe scenarios encountered during driving. At any moment, the AGV will be in a particular context. Fig 3. Data flow in a subset of the current architecture. In regard to the 3–layer architecture, most of the control requirements for the UC resemble those of the previous DARPA challenge, so we will say little about them. Similarly, the deliberative requirements center mainly around planning a route from the MDF and RNDF, and then dynamically replanning if the route is blocked. Path planning in complex situations such as free zones with obstacles also has a deliberative component. The former problem is amenable to standard graph search so we will only discuss how this task integrates with the rest of the system. We have already solved the latter task in the previous challenge [Maid, Lakh], so we shall say little about that as well. Team CajunBot – Source Selection Sensitive Therefore, the focus of the proposal is on the following: situation awareness, and situationappropriate traffic behavior, and sequencing of actions to implement this behavior. Fig 3 shows the data flow architecture of the AGV as currently configured. Before commencing, we provide background on the reliability and safety features of Ragin’Bot’s software and hardware design as it operates autonomously [Lakh]. Ragin’Bot is outfitted with a cluster of computers on an Ethernet. All the sensors emit data to the AGV’s Ethernet. Thus, the drivers for reading these sensors can be run on any computer attached to the Ethernet. The Middleware module, CBWare, provides the infrastructure for communication between distributed processors, so that the producers and consumers of data are independent of each other. We call this a blackboard. Besides providing the usual interfaces to a queue, CBQueues also provides an interface to find in a queue two data items produced at approximately the same time. This ability, made possible due to temporal ordering of data within queues, provides support for an early phase of sensor fusion that is based on the time of data production. Additional sensor fusion further down the processing pipeline is discussed later in the proposal under the topic of the sensor fusion network (SFN). Besides containing descriptors for the immediate situation, the blackboard also contains information about the system state, the AGV’s current activity context and goals, and also state information about the environment. An example of why this last category of information is needed shows up at intersections. The AGV must exhibit proper queuing behavior at an intersection. To achieve this, it must have a memory of which cars have reached the intersection after its own arrival. 2.2. Reliability and Fault Tolerance From a reliability viewpoint, the onboard computers fall into two classes. First, there is the computer that drives the steering controller. Special hardware will monitor this computer’s software heartbeat. If this computer goes down, the hardware will bring the vehicle to a safe stop and restart the computer. If any of the other computers fail, the vehicle shall enter a pause state. After pausing, the machine will be power cycled. If this does not restart the computer, the processes that were running on that machine will be transferred to another computer. Thus, the system will continue to function, but in a degraded fashion because of the higher system load. Regarding software, if a process fails, then we shall restart the process. If a sensor fails and its field of view is covered by another sensor, then the AGV can still operate in degraded mode, as long as the SFN is aware of the failure. System software will also power cycle the sensor in an attempt to revive it. We now address the new requirements posed by the UC. 3. General Approach The UC requires the AGV to exhibit situation-appropriate behavior in different contexts. Within a context, a web of behaviors link together. Lane following linked-behaviors differ from those required at intersections that in turn differ from those required in free zones. We use the term activity context to distinguish these situation categories. The UC problem has four top-level activity contexts that are themselves decomposed into sub-contexts. The top-level contexts are: road following, manage intersections, travel in free zone, and parking. These contexts give fairly complete, non-overlapping coverage of the problem. A solution to one of these categories should not break a solution to another category. Thus, solving these separate problems independently Team CajunBot – Source Selection Sensitive should yield a reasonably complete solution to the UC. (It may be necessary to include other Fig 4. Proposed sensor placement FOVs. Left: Lidar configuration. Region A shows vertically mounted units panning in a 90 sweep. B and C show fixed, horizontally mounted Sick lidars. Middle: Radar configuration. Region A shows movable radars with a 90 sweep and 6 outward bias. B shows a fixed unit with a 12 horizontal sweep. Right: Camera configuration. activity contexts, such as leaving the starting chute.) The UC also requires context specific obstacle avoidance modes. For instance, an approaching vehicle in an opposing lane may show a trajectory that comes close to our own AGV. However, as long as that vehicle’s behavior is laneappropriate, it should not trigger an obstacle avoidance mode in our own vehicle. In a free zone, the required behavior is different. Even within an activity context, a great variety of situation-appropriate behaviors are needed depending on the sub-context. The road following context may be decomposed into the subcontexts of traveling on one-way, two-lane, four-lane roads (or traveling on dirt, primary, and secondary roads). The context of the AGV is readily apparent from the RNDF file, the MDF file, and the vehicle’s position. Each context will have primitive sensor query and primitive actions implemented in the control layer. For instance, in the “navigate a four way stop” context there may be a primitive sensing action to answer the query: “Is a certain position at the intersection free?” Similarly, a primitive action in this context may be to “make a right turn through a clear intersection.” The normal driving rules and experiences will be used to identify an initial set of queries and actions for each context, which will then be tied together using a state machine and debugged in a traffic simulator. A situation is an instance of a context. Situation-action rules apply tests to check for a specific situation where some behavior is either required or safe and appropriate. If the test succeeds, the rule issues a directive to the path-planner (PP), or another appropriate control system to perform an action. In short, our design for the AGV software extends our already existing route following, obstacle-avoidance path-planner with a rule-based system to encode traffic rules and driving knowledge. In order to accurately classify the current situation, the tests invoked by the situation-action rules must reflect the complexity of the situations that must be recognized. The UC poses new technical challenges in areas of situation awareness in a dynamic environment. To meet these challenges, we must: Team CajunBot – Source Selection Sensitive 1. Enhance the physical sensor configuration of the current AGV and enhance the processing of information obtained from the sensors to obtain a richer vocabulary of situation descriptors. 2. Encode rules for situation-appropriate traffic behavior. 3. Manage the complications that arise from the need to fuse information from different sensors. Combining these three issues with the four activity contexts gives a 3 by 4 matrix of subproblems that must be solved to succeed in the challenge. The proposal focuses on these three issues in the context of lane following. The next three sections address each of the issues. 3.1. Enhancing the Sensor Configuration Regardless of the amount of intelligence embedded in the vehicle software, the AGV cannot meet the DARPA UC objectives without adequate (360) sensor coverage to identify objects and their trajectories. Before deciding on the sensor configuration, we cataloged the situationawareness requirements for the various tasks: operating in free zones; intersection-appropriate behavior; passing stopped or slow vehicles; lane-following in primary, secondary, and dirt roads; as well as parking; backing up, making U-turns, and identifying curbs. This analysis converged on the following options for sensor selection and placement to handle the scenarios specified in the UC Rules. The proposed configuration is shown in Fig 4. Fig 4 (left) shows the proposed placement of the Sick (LMS291) lidar. Its primary purpose is for static obstacle detection but a secondary purpose is for detecting dynamic obstacles. Further, we have developed for the last Grand Challenge lidar signal processing and texture analysis algorithms that can classify terrain type (e.g. surface roughness index) and identifying roads with physically defined boundaries (i.e. curbs, ditches, medians). Five units will be mounted on the vehicle. A pair of vertical panning units shall be placed on the front corners of the roof rack for forward motion, obstacle avoidance, lane changes, intersections and other functions. Their range will be about 40 m. Their fields of view (FOV) are labeled A in the figure. A pair of fixed lidar units will be mounted near the doors of the AGV. Their FOVs are labeled B. The B lidar units will be used to identify obstacles when parking and lane changing maneuvers are needed. Finally, a lidar unit, whose FOV is labeled C, will be placed on the rear bumper of the AGV for maneuvers that back up. Fig 4 (middle) shows the placement and FOVs of three Doppler radar units. Their purpose is to detect moving vehicles, their trajectory of approach, and their speed up to a range of Fig 5. Example passing situation. 100m. The center unit will have a fixed mount with the FOV labeled B, which is a 12 wedge. The side units, whose FOVs are labeled A, have moveable mounts. Their default position will be conjoined with region B to give a forward FOV of 36 total. Fig 4 (right) shows FOVs of three cameras. There is a color, close-range camera to identify lane markings and road boundaries. There is Team CajunBot – Source Selection Sensitive also a long-range, monochrome, stereo camera, with a global shutter and hardware exposer and gain controls. These are mounted on the front of the vehicle roof rack to augment the obstacle detecting abilities of the lidar units and provide fault tolerance. In addition to object detection, the RGB / HSI clustering algorithms fused with camera-derived texture and lidar-based roughness information will be used for robust identification of road markings and boundaries. Finally, motion analysis algorithms (optical flow) applied to successive camera frames will identify moving vehicles, their trajectories, and track them from frame to frame. Although, Doppler radar will be the primary sensor for the detection and tracking of moving objects, video data will also be used since it has a wider field of view. A low-level strictly reactive lane following algorithm will be used to minimize dependency on possible weak GPS and/or low accuracy waypoints. 3.2. Encoding Situation-action Rules The rule base for the situation-appropriate traffic behavior must interact with a deliberative system and a waypoint following path planner (with associated steering control). To discuss the interaction with the deliberative system, let us review the static goal-structure of the problem and the issue of dynamic replanning. This provides context for integrating the rule system with the deliberative system. The DARPA-provided MDF lists a series of checkpoints to be visited in sequence. In the AGV, this is represented as a checkpoint queue. Searching the RNDF will yield a waypoint sequence that, if followed, will reach the first, or active, checkpoint. If, at runtime, the AGV discovers that this path is blocked, then the RDNF graph shall be updated to code the blocked route and the updated graph searched to find a new waypoint sequence to reach the active checkpoint. In addition, this new sequence must have enough information to specify the correct approach direction for that active checkpoint. Fig 6. A transition net describing set of rules (used for illustration only). The rules interoperate to generate sequential behavior in a passing situation for a road-following activity context. The number of arcs leaving a state indicates the number of rules that apply to the state. The numbers on the arcs leaving a state indicate the relative priority of those rules. The arc in bold corresponds to the rule in Fig 7. Within this relatively clean framework, complications arise because the AGV’s goal structure must provide safe, situation-appropriate behavior while following this route. Besides avoiding obstacles as it follows waypoints, the AGV must obey traffic rules and avoid the trajectories of moving obstacles. The PP (and supporting control system) is trying to navigate the AGV to the next waypoint while periodically querying (say 1 Hz) the rule base for a new directive. If the PP Team CajunBot – Source Selection Sensitive doesn't receive a new directive, it continues following the previous directive (unless a reactive emergency arises, such as an unexpected obstacle). The blend of reactive and deliberative requirements can be met using the 3-layer design described earlier. We now focus on the situation-specific, sequential behavior requirement. The discussion centers on the passing situation shown in Fig 5. At any given time, the AGV’s actions are determined by its current goals and model of the current situation. For instance, the AGV in Fig 5 will notice (via sensor fusion applied to the GPS, RDNF, and sensor input) that it is in a two-lane, primary road with a stopped vehicle immediately ahead. Passing the vehicle involves a sequence of actions that depend on ongoing sensor input. In this and the next section, we address the sequencing requirements and the sensor interpretation and fusion requirements of this problem. Fig 7. Rule to begin passing in 2-lane segment. name: roadFollowing.2lanes.stopped tests: 1. “AGV in home lane” 2. “AGV not moving” 3. “static obst less than 8m ahead” 4. “dist from exit_wp gt 40m” 5. “opposing lane clear fore and aft” state update: “passing stopped object” “starting to pass” action: “enter opposing lane against flow” There should be a situation-action rule to direct the AGV to pass a stopped object. A passing action, however, is not an atomic action. It has subcomponents that are executed sequentially and it is embedded in a web of behaviors associated with road following. We also need to decide on the grain size of a situation. Our approach is to choose a situation grain size small enough so that the behavior grain size is sufficiently small. By sufficiently small we mean, the action required in the situation is understandable to our path planner and the path planner can then implement the directive while operating in reactive mode. The state-transition network in Fig 6 shows a small web of behaviors specified by a set of situation-action rules. Each node in the diagram is a situation. The situations arise in a road following context where the AGV encounters a stopped vehicle. The number of edges leaving a node indicates the number of rules that apply to that situation. Thus, an edge corresponds to a rule. The rules that apply to a situation have slightly different tests to discriminate which rule should apply in the situation. The arcs leaving a node have a number to indicate the relative priority of the corresponding rules for that node. If tests on two separate rules for that situation are satisfied, the rule with the higher priority is applied. This eliminates ambiguity in the rule base and, more importantly, makes it easier to implement a form of negation as failure. For concreteness, we introduce an example situation-action rule that initiates the action of passing a stopped vehicle. The rule appears in Fig 7, applies to the traffic situation in Fig 5, and participates in the set of rules depicted in the state transition diagram shown in Fig 6. The rule has three parts: 1) a condition that recognizes a situation; 2) a sequence of state update actions that code the system’s activity context; and, 3) a path planner (PP) directive which tells the planner to take some named action. When the PP executes a directive, the AGV actions generally lead to a new situation and updated internal model. This new situation provides different sensor input. In this new situation, the current rule will no longer match the situation and a different rule in the rule base will apply. The informal state transition diagram in Fig 6 shows how a system of rules interoperates to generate a sequence of actions. The rule base has the following features. Team CajunBot – Source Selection Sensitive 1. The rule-base shall be hand crafted by the software team. The content of the rulebase shall be debugged in a traffic simulator. This will allow us to maximize the completeness of the rules, detect any unanticipated interactions, and also get an idea how the system might behave in unanticipated situations. 2. The rule language is simple so that the rule-base can be transformed and optimized into executable source code that implements a decision tree. 3. The rules that might apply to a particular situation are prioritized. 4. A rule is a situation/action pair. The situation is a sequence of tests and the action is one or more directives. An action can be a directive to the path-planner (PP) and/or a directive to change the goal state. Executing a rule causes the system to execute a directive. Executing the directive yields either a new goal state or leads (by sideeffect) to a new situation, in which a new rule may apply. If no rule applies, the PP continues its default reactive behavior. 5. The tests are applied to the real time blackboard. The blackboard handles sensor fusion and reasoning under uncertainty. The tests in a rule are evaluated sequentially using conditional evaluation (if a test fails, remaining tests are not evaluated). Let us briefly comment on the rule in Fig 7. The context can be established by comparing the GPS information with the contents of the RNDF. The remaining tests name source code functions which apply tests to an internal model. The tests vary in complexity. The most difficult test (Test 5) is listed last because it may require physical movement of the sensors and will need to detect moving objects, their trajectories, and speeds at a long distance (approx 100 m). The conditional evaluation protocol ensures that this test is not performed unless absolutely necessary. The action field consists of one directive to the PP that is responsible for local path planning, reactive obstacle avoidance, and PID-based steering control. The PP shall understand some vocabulary of directives. Directives are simple enough so that the PP can generate and follow a series of local waypoints going into the opposing lane. Fig 8. Distributed-memory, blackboard implemented in middleware to facilitate sensor fusion and reasoning under uncertainty. Since the semantics of the tests, are grounded in descriptors in the internal model that are extracted from distributed sensor readings across different sensor modalities, we postpone further discussion of the tests until the next section. To address the robustness of the approach, let us consider unanticipated situations. What happens if the AGV miscalculated the distance of oncoming vehicles and an oncoming vehicle appears in the opposing lane? Since the vehicle in the home lane is stopped and the AGV would already have returned to the home lane if it weren't blocked, its only option is to either stop or veer to the left leaving the road. Either of these options can be handled by the reactive mode of the PP. Another unanticipated event is that, while passing, the Team CajunBot – Source Selection Sensitive obstacle that was stopped turns out to be a vehicle that begins moving. As long as the vehicle does not travel at exactly the same speed as the AGV, there is no other traffic, and the AGV is sufficiently far from an intersection, the home lane will eventually become clear. At that time, the rule base will direct the planner to return to the home lane. In this latter example, the behavior given by this small, illustrative rule system is not acceptable for UC’s advanced traffic requirements. Thus the rule system needs to be extended for situations that pass a moving vehicle. 3.3. Sensor Fusion and Reasoning Under Uncertainty Returning to the rule in Fig 7, the semantics of the tests are grounded in descriptors in the internal model. These descriptors, for the most part, derive from the Fig 9. SFN information source diagram. Nodes represent estimates of values of continuous variables (mean and readings of many sensors over confidence interval). Edges entering a node represent different modalities. In our current conditional probability tables. implementation, the output of the sensors is written as situation descriptors to a blackboard implemented in middleware as a virtual shared memory. We propose to add an approximate Bayesian inference network, or some variant, as a back end to this blackboard and we call the combined system a sensor fusion network (SFN). This is shown in Fig 8. The SFN must operate with continuous variables and be compatible with Kalman filters [Russ, Neap]. To make the requirements more explicit, reconsider evaluating the tests in the example situation-action rule of Fig 7. Consider what is needed to perform the third test: Is a static obstacle less than 8 m ahead of the AGV. The most reliable information source for this test is the lidar. While the AGV is moving, radar is also relevant as a secondary source. Finally, stereo vision is relevant as a tertiary source. The information from these sources can be combined if: 1) There is a causal model that accounts for discrepancies in the interpretation of the situation (e.g., Information that radar only provides information on obstacles moving relative to the vehicle); 2) The model can be expressed as conditional probabilities; and, 3) The scene descriptors provided to the SFN are given in the language of event probabilities, estimated means, and confidence ranges. Let us provide more detail about what is expected from the SFN. Consider the problem of estimating the AGV’s position within a lane. The AGV’s positional coordinates are estimated on the basis of GPS/INS, lidar, and vision. If the GPS becomes unstable, then the AGV shall place more emphasis on alternative information sources such as the lidar and vision. If the GPS unit estimates its error, then the SFN can shift the emphasis. Fig 9 shows the details of causal the model. The output node of the SFN is the AGV’s location in a lane. For this illustrative example, it is a continuous scalar variable represented as a confidence interval. Its value is determined by Team CajunBot – Source Selection Sensitive three information sources: the GPS location estimate (combined with the RNDF data about lane locations), knowledge of the location of the center boundary, and the right boundary. The arcs impinging on this node specify how to combine the information from the three information sources. The other nodes in the network have an analogous interpretation. If the GPS-sat readings become unreliable, then the GPS location estimate must depend only on the INS, which must depend on its acceleration readings and input from the wheel encoder. Therefore the confidence intervals for the GPS location estimate must widen and the downstream outputs of the network will be accordingly affected (the network will not have cycles). For advanced navigation, there is an issue of navigating roads that have inaccurate waypoints in the RNDF. If such a waypoint is inaccurate, then it is still useful while the AGV is far away from the waypoint. This is because the useable level of detail required for distant objects is lower. However, the low-accuracy waypoint is not useful if the vehicle is right next to it. We do not have a solution to the problem at the moment, but we show how discussion of the problem can be sharpened in the context of the SFN information source diagram. Normally, the RNDF data is considered accurate. To solve this problem, we need a way to when to widen the confidence intervals output by the RNDF data node in Fig 9. 4. Summary and Closing The above narrative presents our technical strategy to address the requirements of the requirements of UC. Our architecture strategy provides us the ability to develop the system in monotonically increasing chunks. The first property follows from our hierarchical context based partitioning of the problem space. Solution of problems in one context will not interfere with the solution of another problem in another context. Thus, each solution can be implemented separately, and the entire system created by composing smaller chunks. This is an important advantage to ensure that the final vehicle will not behave in an unpredictable way due to interactions between solutions to different problems. 5. References [Gat] Gat, E. Three-layered architectures. In D. Kortenkamp, R. P. Bonasso, and R. Murphy(Eds), AI-based Mobile Robots: Case Studies of Successful Robot Systems, p. 195210, Cambridge, MA: MIT Press, 1998. [Felz] Felzenszwalb, P. F., Huttenlocher, D. P. Efficient graph-based image segmentation. Intl. Journal of Computer Vision, 2004, 59(2). [Lakh] Lakhotia, A., Golconda, S., Maida, A, Mejia, P., Puntambekar, A., Seetharaman, G., Wilson, S. CajunBot: Architecture and algorithms. Journal of Field Robotics, in press. [Maid] Maida, A., Golconda, S, Mejia, P., Lakhotia, A., Cavanaugh, C. Subgoal-based local navigation and obstacle avoidance using a grid-distance field. International Journal of Vehicle Autonomous Systems, in press. [Neap] Neapolitan, R., E., Learning Bayesian Networks. Upper Saddle River, NJ: Prentice Hall, 2004. [Russ] Russell, S., Norvig, P. Artificial Intelligence: A Modern Approach (second edition). Upper Saddle River, NJ: Prentice Hall, 2003.
© Copyright 2026 Paperzz