Intelligent Control Approaches for UAVs - Intelligent Systems Division [PDF]

Intelligent Control Approaches for UAVs

K. KrishnaKumar NeuroEngineering Laboratory NASA Ames Research Center

Presented at UAV-MMNT03 K. KrishnaKumar

NeuroEngineering Lab

Presentation Outline

• Intelligent Control Background • Intelligent Flight Control Research @ NASA Ames

K. KrishnaKumar

NeuroEngineering Lab

• Intelligent Control Background – What are intelligent systems – What is intelligent control – Intelligent control architectures

K. KrishnaKumar

NeuroEngineering Lab

Defining Intelligent Systems ¾ An Intelligent System is one that exhibits any of the following traits: 9 Learning 9 Adaptability 9 Robustness across problem domains 9 Improving efficiency (over time and/or space) 9 Information compression (data to knowledge) 9 Extrapolated reasoning

IS is seen as Rationalistic AI: Intelligence for doing the right thing K. KrishnaKumar

NeuroEngineering Lab

Intelligent Control U_desired ?

Y_desired ?

System

Control U

Y

Two Error Signals are needed: 1. System Performance Error Signal 2. Control Error Signal

K. KrishnaKumar

NeuroEngineering Lab

Questions How do we say that one controller is more intelligent than the other? Can the intelligence be improved? Can intelligence be measured?

Answer : Levels of Intelligent Control

K. KrishnaKumar

NeuroEngineering Lab

Levels of Intelligent Control System Features

U +_

System or Plant

Y

Mission Planning Trajectory Optimization Flight Path Adaptive Control Flight Path Stabilization

Level 0

Y desired

Level 1 Level 2 Level 3

L ev 0

Self im provem ent of: T rackin g E rror (T E )

1

T E + C on trol Param eters (C P)

2

T E + C P + Perform an ce M easure (PM )

3

T E + C P+ PM + Plan n in g Fun ction

K. KrishnaKumar

D escription Robust Feedback con trol (E rror ten ds to zero). Robust feedback con trol with adaptive con trol param eters (error ten ds to zero for n on -n om in al operation s; feedback con trol is self im provin g). Robust, adaptive feedback con trol th at m in im izes or m axim izes a utility fun ction over tim e (error ten ds to zero an d a m easure of perform an ce is optim ized). Level 2 + th e ability to plan ah ead of tim e for un certain situation s, sim ulate, an d m odel un certain ties. NeuroEngineering Lab

Levels · Level 0: Robust stabilization • • • • •

Gain Scheduling Supervised neuro-control Fuzzy control Mimic a controller Implicit Control

K. KrishnaKumar

NeuroEngineering Lab

Levels · Level 1: Adaptive Control • • • •

K. KrishnaKumar

Learn Systems and Controller Parameters Neural adaptive Control Adaptive inverse Control Approximate Controller error signal

NeuroEngineering Lab

Levels · Level 2: Optimal Control • • • • •

Reinforcement Learning Control Allocation Dynamic programming Linear Adaptive Critics Non-linear Adaptive Critics

y(t+1)

Aircraft + Controller

yd(t)

up(t)

Reference Model

Π e(t),up(t) e(t+1), etc

e(t), etc

K. KrishnaKumar

Critic (t+1) Critic (t)

λ (t+1)

λ (t)

дU(t)/д e(t) γ(1-ωdt)

+

+

NeuroEngineering Lab

Levels · Level 3: Planning Control (More AI-like) • • • • • • •

Strategic Planning Strategic search Mission Planning HTN: hierarchical task network Production-based cognitive architectures Decision-theoretic (MMDP) Etc..

FAILURE EVENT

Systems ID Updated Flight Dynamics / Performance Boundaries LANDING SITE SEARCH

ASAC AIRPORT DATABASE

Airport List

Sorted Feasible Airport List TRAJECTORY PLANNING Trajectory

GN&C

K. KrishnaKumar

NeuroEngineering Lab

NASA Ames Intelligent Flight Control Applications

K. KrishnaKumar

NeuroEngineering Lab

Manned Aircraft Objectives Develop flight control technologies that can automatically compensate for problems or failures when they occur Develop these technologies and capabilities in a generic sense so that they can be applied to different vehicle classes Application Platforms – B 757 class aircraft – Simulation only – F-15 – In Flight test – C-17 – Flight tests in 2004

K. KrishnaKumar

NeuroEngineering Lab

Pre-Trained Neural Networks Step 1 Integrated Vehicle Modeling Env. Rapid Aircraft Modeler (RAM)

K. KrishnaKumar

Step 2 Vortex Lattice Code (VORVIE W) Mass/Inertia Estimates (Balance)

Step 3 Levenberg Marquardt neural net Optimal Pruning Algorithm

NeuroEngineering Lab

Neural Flight Control Architectures Adaptive Critic technology Generalized control reallocation Adaptive Critic

Level 2 pilot inputs

Desired Handling Qualities Reference Model

+

Optimal Control Allocation

Sensors

Controller

Direct Adaptive “on-line” Neural Network

Level 1

Indirect Adaptive & “pre-trained” Neural Network(s)

Level 0 stability & control derivative estimates

-

K. KrishnaKumar

NeuroEngineering Lab

Level 1 Adaptive Control Equations

K. KrishnaKumar

NeuroEngineering Lab

Level 1 Control Equations

K. KrishnaKumar

NeuroEngineering Lab

Level 1 Control Equations

K. KrishnaKumar

NeuroEngineering Lab

Level 2: Optimal Control Allocation • When to allocate? – Control limit violation – Rate saturation – Control failure • How to allocate? – Optimal allocation using Linear Programming • Conventional hierarchy • Best available hierarchy

K. KrishnaKumar

NeuroEngineering Lab

Example Aerodynamic Control Authority Directional Authority

aileron_left

aileron_right

rudder_upper

rudder_lower

elevator_lob

elevator_lib

elevator_rib

elevator_rob

spoiler_lib

spoiler_lmib

spoiler_lmob

spoiler_lob

spoiler_rib

spoiler_rmib

spoiler_rmob

spoiler_rob

Pitch Conrol Authority

K. KrishnaKumar

Roll Control Authority

NeuroEngineering Lab

Linear Programming Formulation Dynamic System is defined as

[X& ] = f ( X ) + [B][u] + f Let us write

[B ][u ]

BUU BUL uU  B B u +∆u  =  LU LL L L

trim

as

BUU BULuU  B B u  +  LU LL L 

BUU BUL 0  B B ∆u   LU LL L

uU = Unlimited Control Vector from Dynamic Inverse u L + ∆u L = K. KrishnaKumar

Limited Control Vector from Dynamic Inverse NeuroEngineering Lab

L P Formulation (cont’d) What we need is help from Unlimited Control

 BUU ∆uU   BUL ∆u L   B ∆u  =  B ∆u   LU U   LL L 

Let us now define a control reallocation matrix [λ ] such that

[∆uU ] = [λ ][∆u L ]

=>

Define a linear relationship

[α ][λ1 K. KrishnaKumar

λ2

 BUU   BUL   B [λ ] =  B   LL   LU 

[α ][λ ] = [β ]

. . λm ] = [β1

β2

. . βm ] NeuroEngineering Lab

LP Formulation (Cont’d) min λi

T

(wi λi )

Subject to

[α ][λi ] ≤ [β i ]

0 ≤ λ i ≤ λ max

and

Example: 4 control inputs  w11 w [W ] =  21  w31   w41 K. KrishnaKumar

w12

w13

w22

w23

w32

w33

w42

w43

w14  w24  = [w1 w34   w44 

w2

w3

w4 ]

NeuroEngineering Lab

Conventional & Best Hierarchies Elevator

Left Aileron

Right Aileron

Primary

Secondary

Secondary

Left Aileron

Primary

Secondary

Tertiary

Right Aileron

Secondary

Primary

Tertiary

Rudder

Secondary

Secondary

Primary

Elevator

Conventional

[W ]

T

 * 100 = 100  100

K. KrishnaKumar

1 1 100 * 1 10  1 * 10   1 1 * 

Rudder

Best [W ]T

 * 100 = 100  100

1 * 1 1

1 100 1 1  * 1   1 *  NeuroEngineering Lab

Implementation • Primary Cost based on “surface”

min T (w u ) u • Auxiliary Cost based on “axis error”

min u

K. KrishnaKumar

T

(c e )

NeuroEngineering Lab

Level 2 Controller

Reference Model Adaptation using an Adaptive Critic Adaptive Critic

pilot inputs

K. KrishnaKumar

Desired Handling Qualities Reference Model

NeuroEngineering Lab

Adaptive Critic Adaptive critic designs have been defined as designs that attempt to approximate dynamic programming.

J (t ) =< γJ (t + 1) > + min U (t ) u X(t) X(t+1)

System Model

u(t)

+

∂J (t + 1) ∂X (t + 1) γ ∂X (t + 1) ∂u(t )

X(t+1)

Critic

+

U(t)

γ

+ γJ(t+1)+U(t)

Critic J(t)

K. KrishnaKumar

∂U (t ) ∂u(t )

J(t+1) 1.0

X(t)

X(t)

Controller

+

NeuroEngineering Lab

Level 2 Control

y(t+1)

Aircraft + Controller

yd(t)

up(t)

Reference Model

Π e(t),up(t) e(t+1), etc

Critic (t+1) e(t), etc

K. KrishnaKumar

Critic (t)

λ (t+1)

λ (t)

дU(t)/д e(t) γ(1-ωdt)

+

+

-

NeuroEngineering Lab

Results for Series of Failures

During tactical descent (failures on one side) · 23,000’: Stab frozen at trim · 20,000’: 2 Elevators frozen at 0 deg. · 17,000’: Upper rudder hard over · 15,000’: Outboard flap fails retracted · 14,000’: Aileron frozen at 0 deg. · 13,000’: Two outboard spoilers frozen at 0 deg. When engines come out of reverse: Outboard engine seizes K. KrishnaKumar

NeuroEngineering Lab

Intelligent Maneuvering of UAVs •Goals –Provide increasingly higher levels of automation, capable of responding to changing goals and objectives, while taking corrective actions in the presence of internal or external events. –Allow pilots, ground-based operators or autonomous

executives to defer the responsibilities of performing and supervising tasks, to focus on managing goals and objectives.

K. KrishnaKumar

NeuroEngineering Lab

Intelligent Maneuvering of UAVs Level 2

Level 3

Strategic Planning

Tactical Maneuvering

Trajectory Trajectory Specialists Specialists

Maneuver Maneuver Database Database

Level 1

Autopilot

Flight Controller

Vehicle

Sensors (IRS) Sensors (ADC)

Sensors (NAV) ADC- Air Data Computer IRS - Inertial Reference System NAV- Navigational System (GPS & Visual Perception)

K. KrishnaKumar

Continuous-Time Commands & Sign Discrete-Time Commands

NeuroEngineering Lab

Flight Controller

K. KrishnaKumar

NeuroEngineering Lab

Tactical Maneuvering Performs time-critical flight path operations, which includes aggressive maneuvers in the presence of unexpected obstacles.

•Inputs –Commands •Reference Targets / Trajectory •Performance Parameters –Awareness •Threat Detection (eg. TCAS, GCAS) •Vehicle Performance Models •Outputs –Maneuver Sequence •Control Law Specific Modes & Targets •Transition Criteria •Maneuver Selection Specialists

Immunized Maneuver Selection Off-line

Memory BB Known Systems, Solutions, Requirements, etc

Learning

Building Blocks

Clonal Selection

On-line Immune Features

Optimal Maneuver

Utility Measures, Performance Criteria, Constraints, etc.

Krishna Kumar

Model Predictive Take-off and Landing

–Immunized Maneuver Selection –Heuristic-Based TSP Maneuver Selection •Maneuver Database –Elements & “Canned” Sequences K. KrishnaKumar

Eric Wan

NeuroEngineering Lab

A system-level description of the Immune System Metaphor

Antigen/Threat/Problem

Memory Information available in the DNA Molecule.

Immune Network Bone Marrow (Model)

•a priori knowledge, •Shape Space •Simulation, System •Representation issues Models, etc. (Binary, etc)

K. KrishnaKumar

Negative Selection

•Self-Nonself recognition •Discrimination •partly in T-cells.

Clonal Selection

•Definition of Antigen •Antigen-Antibody strength (fitness) definition

NeuroEngineering Lab

Tactical Maneuvering Database Contains general and aircraft specific maneuvering database elements, each corresponding to associated control laws. Pre-canned maneuver sequences represent domain expertise. •Elements –Control Law •Mode & Target Definition –Aircraft Specific •Flight Envelope Validation Logic –Specifications •Closed-Loop Predictive Models (x0, …, xf ) •Resource Allocation Table (e.g. lat, lon, ped, thr/col) •Sequences –Elements •Specified Parameters / Arguments –Transition Criteria / Termination Logic •Time-Based and/or Condition-Based –Interrupts •Abort Conditions & Abort Sequence K. KrishnaKumar

Bank to Turn Element Heading Select (coord. turn) Mode: HDGSEL Target: Heading = [arg1] deg Envelope: IAS > 180 kts, |θ| < … Model: ϕo/ϕi = τ /(τ s+1), ϕ‘max = gcos(θ)sin(φmax)/vt RAT: LAT/PED

Bank & Pull to Turn Sequence Bank Left: 0 Normal Accel. (speed control) Bank Right: +90 Mode: BANKSEL Target: Bank = 90 deg, Vz = Vz0 Envelope: mach > 0.4, |α| < … Model: φo/φi = τ /(τ s+1), φ‘max = pmax Vz = Vz0 RAT: LAT/PED

NeuroEngineering Lab

Autopilot System (Example) Vertical Speed From Altitude

FPA From Airspeed

Vertical Speed From V-Path

FPA From Vertical Speed

Pitch Rate From FPA

Fixed Vertical Speed

Fixed FPA

Pitch Rate From Body-Axis Pitch

Longitudinal Stick

Thrust From Vertical Speed Thrust From Airspeed Turn Rate From Heading

Throttle

Fixed Thrust

Turn Rate

•Longitudinal Modes –Pitch, Nz, AoA, FPA; Mach, IAS, Vertical Speed; Vertical Path, Altitude •Thrust Modes –Mach, IAS, Vspd, Thrust; Vertical Path, Altitude, FPA •Lateral Modes –Bank, Roll Rate; Heading, Track; Lateral Path •Directional Modes –Sideslip, Ny, Heading K. KrishnaKumar

NeuroEngineering Lab

Results 7500

Altitude (ft)

7000 6500 6000 5500 5000 4500 1

4

x 10

x 10

-1

-Y (ft)

-2

-4000

-2000

0

X (ft)

2000

4000

6000 1.6 1.5

Altitude (ft)

0 4

1.4 1.3 1.2 1.1

Three Chained Modes

1 1 0.5 4

x 10

2 1

-0.5

-Y (ft)

K. KrishnaKumar

1.5

0 -1

0.5 0

4

x 10

X (ft)

NeuroEngineering Lab

Strategic Maneuvering Performs long-term planning that meets dynamic mission goals and objectives, within mission constraints and performance limitations.

K. KrishnaKumar

Low-Altitude Energy Management 15

10

CROSSTRACK (nm)

•Inputs –Goals •Cost Function •Mission Constraints –Awareness •External Obstacles (weather, terrain, …) •Internal Health & Performance Limitations •Outputs –Extended Flight Plan •Waypoints / Reference Trajectory •Performance Parameters •Configuration Schedules •Trajectory Specialists –Energy Management Guidance •Tear Drop, Low-Altitude, Enroute –Evolutionary Navigation

5

0

TextEnd

-5

-10

-15 -20

-15

-10

-5 0 DOWNRANGE (nm)

5

10

15

John Bull

Obstacle Avoiding Evolutionary Navigation

NeuroEngineering Lab Accurate Automation Corporation

Optimal Way Point Computation Around Obstacles Using Evolutionary Algorithms ¾The Algorithm: ¾Step 1: Determine the obstacles that are in the path of the flight ¾Step 2: Place the waypoints for the aircraft on the circumference of the obstacles ¾Step 3: Compute the path between the start and the end using the waypoints. ¾Step 4: Compute a fitness function ¾Step 5. After “n” iterations the best set of waypoints defines the navigation path.

K. KrishnaKumar

NeuroEngineering Lab

Demo

K. KrishnaKumar

NeuroEngineering Lab

Intelligent Control for BEES Exploration of Mars using Free-flyers with sensors inspired by Nature Mars Lander Mars Flyer

Controller Objectives: ¾Maintain safe distance from the Lander and ensure local stability. ¾Point in the desired attitude and follow a trajectory to enable imaging of interesting Geological Picture. ¾Optimize long-term and short-term goals, such as minimization of fuel (long-term) and avoid collision with the Lander (short-term) ¾React to changing environments by adapting the control functionality

K. KrishnaKumar

NeuroEngineering Lab

Pre-Trained Neural Networks Step 1 Integrated Vehicle Modeling Env. CAD Designs

K. KrishnaKumar

Step 2 Cartesian Euler Code (CART3D)

Step 3 Levenberg Marquardt neural net Optimal Pruning Algorithm

NeuroEngineering Lab

Level 2 Architecture

Accn Commands

+ ARS Features

Tactical Maneuvering

P + I Error Controller

++

Strategic Planner

Dynamic Inversion

Learning Neural Network

Pretrained NNs

Control Allocation

Sensor Suite

K. KrishnaKumar

NeuroEngineering Lab

Concluding Remarks 9Intelligent control comes in many flavors 9Levels of Intelligent Control is one way of quantifying the roles of Intelligent control 9Intelligent control architectures allow for fast prototyping 9Intelligent control architectures can guarantee inner-loop stability 9For UAV application, intelligent control provides a robust way to accommodate any outer-loop architecture for planning, etc.

K. KrishnaKumar

NeuroEngineering Lab