Making the Most of Your Resources: How to Design Simulations and Experiments to Achieve Your Engineering Goals Ilias Bilionis
[email protected] School of Mechanical Engineering Purdue University predictivesciencelab.org 1
Predictive Science Laboratory Ph.D students:
Funding:
2
Predictive Science Laboratory Research •
Uncertainty quantification.
•
Human-machine interaction (buildings).
•
Sociotechnical modeling (smart and connected cities)
•
Systems Science (modeling the human aspect of systems engineering processes). can talk offline… 3
Predictive Science Laboratory - UQ Research Problems: •
Propagate uncertainty through models.
•
Characterize sensitivity to inputs (models or experiments)
•
Calibrate models on experimental data.
•
Design optimization under uncertainty.
Focus: •
Limited experimental and computational budget.
•
High-dimensional uncertainties/design variables.
•
Information fusion (fuse multi-fidelity models and experiments). 4
Example 1: Quantification of Mass Production Uncertainties in Electric Machines Fig 4.10. Flux density map in base SRM (left) and design P2 (right) at
, with the
currents corresponding to the generating mode
Uncertainties in electric machines: Fig 4.11 Flux density map in base SRM (left) and design P2 (right) at , with the • Geometries (manufacturing imperfections). currents corresponding to the generating mode • Magnetic properties of materials (supply chain changes). • Operating conditions (extreme environments or future application challenges).
Fig 3.7. Speed, torque, and power of the electric machine as a function of time
Fig 3.8. Speed, torque, and power of the electric machine for one start-stop operati
Dionysis Aliprantis Associate Professor ECE, Purdue
5
It can be observed from Fig 3.8 that when the bus begins to accelerate (as obs
Example 2: Stochastic optimization of wiredrawing process f (FEM simulations) Pass 1
Pass 2
…
Pass n Outputs
Process Controls
Reduction Ratios
Die Angles
Ultimate Tensile Strength
Strain nonuniform factor
max
min
Uncertainty
Jitesh Panchal ME Purdue
Piyush Pandita Phd Student
6
Example 3: Optimization of Roll-to-Roll Plasma Systems for High-quality Graphene Growth Process Controls
Outputs Raman Spectroscopy
R2R-PE-CVD
Gas Conc/tions
Tim Fisher UCLA
Uncertainty Pressure
Plasma Power
Piyush Pandita Phd Student
Majed Alrefae Phd Student
7
D/G Intensities (Right)
D/G Intensities (Left)
Example 4: Characterization of Sensitivity Indices from Experiments Input Parameters
$10,000 experiment
Quantities of interest
Problem: Select experiments to optimally infer the order of the Sobol sensitivity indices under given budget constraints. Jesper Kristensen GE global research 8
Mathematical Abstraction Input Parameters
Physical model/ Experiment
Quantities of interest
x
f
y
We’ll think about it as a mathematical function: y = f (x)
9
Mathematical Abstraction •
Uncertainty propagation: f
p(x )→ p(y ) •
Model calibration: f
y,p(x )→ p(x | y ) •
Design optimization under uncertainty: x = max Eξ [O(f (x; ξ ))] *
10
Monte Carlo methods? Yes, but… The Statistician (1987) 36, pp. 247-249 247
Monte Carlo is fundamentally unsound
A. O'HAGAN
Department of Statistics, University of Warwick, Coventry CV4 7AL, U.K.
Abstract. We present some fundamental objections to the Monte Carlo method of numerical tion.
1 Background
11
The Surrogate Idea •
Do a finite number of simulations.
•
Replace model (experimental response surface) with an approximation: y ≈ fˆ(x)
•
The surrogate is usually cheap to evaluate.
•
Solve the UQ problem with the surrogate. 12
Issues of (Classic) Surrogates •
curse of dimensionality
•
need a lot of simulations/experiments
•
…
Limited simulations/experiments increase epistemic uncertainty!!! 13
How do we quantify epistemic uncertainties due to limited observations?
14
The Bayesian approach •
Put probability on response surfaces (prior).
•
Evaluate response on a finite set of inputs.
•
Use Bayes rule for posterior on response surfaces.
•
Harness epistemic uncertainty solve UQ problems.
Most people, even Bayesians, think that this sounds crazy when they first hear about it. -Persi Diaconis (1988)
15
The Bayesian surrogate idea Space of response surfaces f: X->Y p(f (⋅))
p(f (⋅) | D)
Bayes rule
Epistemic uncertainty Before I see data (prior)
16
After I see data (posterior)
Uncertainty Propagation with Limited Data (c) 33
Prediction for variance
Input uncertainty
Take variance x (b)
V[y] (e) Epistemic uncertainty due to limitedSubfigure simulations Fig. 1. Synthetic: (a) shows the ob 17
Example 2: Stochastic optimization of wiredrawing process f (FEM simulations) Pass 1
Pass 2
…
Pass n Outputs
Process Controls
Reduction Ratios
Die Angles
Ultimate Tensile Strength
Strain nonuniform factor
max
min
Uncertainty
Jitesh Panchal ME Purdue
Piyush Pandita Phd Student
18
Example 2: Stochastic Multiobjective Optimization max x E[f1(x,ξ )] max x E[f2 (x,ξ )]
Pareto Front (set of non-dominated designs) What can you say about the Pareto front if you can run 20 simulations?
19
Example 2: Multi-objective Optimization
Visualization in objective space
Median (predicted) Pareto front
Epistemic uncertainty (predicted) Pareto front
Observed simulations
20
Example 3: Optimization of Roll-to-Roll Plasma Systems for High-quality Graphene Growth Process Controls
Outputs Raman Spectroscopy
R2R-PE-CVD
Gas Conc/tions
Tim Fisher UCLA
Uncertainty Pressure
Plasma Power
Piyush Pandita Phd Student
Majed Alrefae Phd Student
21
D/G Intensities (Right)
D/G Intensities (Left)
D/G
Example 3: Optimization of Roll-to-Roll Plasma Systems for High-quality Graphene Growth
Pressure (mbar) 22
of a mode. is to n the e the and trapd on
Left side ID/IG
aims R2R es.
Example 3: Optimization of Roll-to-Roll Plasma Systems for High-quality Graphene Growth
Right side 23 ID/IG
How do we select which simulation/experiment to run next? Epistemic uncertainty is the key to constructing information acquisition algorithms! 24
Select Query of Max Epistemic Uncertainty
Not universally optimal! 25
What if we want to learn more about something specific, say quantity Q? Where Q could be: • expectation/variance of output quantity • calibrated model parameters • optimal design/control/value • order of sensitivity indices • probability of rare event • … 26
General idea 1 Collect data in order to maximize the expected information gain about Q. What we would think about Q if at x we observed y What we currently think Q is.
Information gain
Q
Take expectation over what we think y could be G(x;Dn ) = ∫ KL(p(Q | Dn ,x,y )! p(Q | Dn ))p (y | x,Dn )dy 27
General idea 2 Collect data in order to maximize the expected value of information. How much value have we obtained so far from our experiments/simulations? V(Dn ) − c(Dn ) How much additional value would we get if at x we observed y? V(Dn ,x,y ) − c(x ) − [V(Dn ) − c(Dn )]
Take expectation over what we think y could be
G(x;Dn ) = ∫ V(Dn ,x,y )p(y | x,D)dy − c(x ) − c(Dn ) − V(Dn ) 28
Generic Information Acquisition Paradigm xn+1 = argmax x G(x;Dn )
A heuristic! •
Real problem: “Find policy that maximizes the expected information gain/value of information.”
•
A stochastic dynamic programming problem.
•
Very hard… but may pay off. 29
Generic Information Acquisition Paradigm Simulation and Experimental Data
Perform new simulation/ experiment
Select Query Maximizing Expected Information Gain/ Value of information
State of Knowledge about Quantities of Interest
Is epistemic uncertainty acceptable?
State of Knowledge about Response Surfaces
No
Bayes rule
30
Yes
Report result
Example: 1D robust optimization True, unobserved, objective to minimize Noisy objective Value of measurements information
p(f (⋅) | Dn )
(Pandita and Bilionis, 2016) 31
Example 3: Optimization of Roll-to-Roll Plasma Systems for High-quality Graphene Growth Process Controls
Outputs Raman Spectroscopy
R2R-PE-CVD
Gas Conc/tions
Tim Fisher UCLA
Uncertainty Pressure
Plasma Power
Piyush Pandita Phd Student
Majed Alrefae Phd Student
32
D/G Intensities (Right)
D/G Intensities (Left)
Example 3 - Graphene Manufacturing Sets 1-3 Set 4 Set 5 Set 6
1.4
Leftside ID/IG
1.2 1.0
50 µm 0.8 0.6 0.4 0.6
0.8
1.0
1.2
1.4
Rightside ID/IG
ion of graphene quality (I D/IG) of the leftside and al method for optimization process where the 33 oved from the previous sets.
50 µm
Uncertainty Propagation with Limited Resources References •
Bilionis, I. and N. Zabaras (2012). "Multi-output local Gaussian process regression: Applications to uncertainty quantification." Journal of Computational Physics 231(17): 5718-5746.
•
Bilionis, I. and N. Zabaras (2012). "Multidimensional adaptive relevance vector machines for uncertainty quantification." SIAM Journal on Scientific Computing 34(6): B881-B908.
•
Bilionis, I., et al. (2013). "Multi-output separable Gaussian process: Towards an efficient, fully Bayesian paradigm for uncertainty quantification." Journal of Computational Physics 241: 212-239.
•
I. Bilionis and N. Zabaras, “Bayesian uncertainty propagation using Gaussian processes,” in Handbook of Uncertainty Quantification, no. 16: Springer International Publishing, 2016, pp. 1–45.
34
Stochastic Optimization with Limited Resources References •
Pandita, P., Bilionis, I., & Panchal, J. (2016). Extending expected improvement for highdimensional stochastic optimization of expensive black-box functions. Journal of Mechanical Design, 138(11), 111412.
•
Kristensen, J., Bilionis I., & Zabaras, N. (2017). Adaptive simulation selection for the discovery of the ground state line of binary alloys with a limited computational budget. Recent Progress and Modern Challenges in Applied Mathematics, Modeling and Computational Science.
•
Alrefae, M., Kumar, A., Pandita, P., Candadai, A., Bilionis, I., Fisher, T. (2017). Process optimization of graphene growth in a roll-to-roll plasma CVD system. AIP Advances.
•
Pandita, P., Bilionis, I., & Panchal, et al. (2018). Stochastic multi-objective optimization on a budget: Application to multi-pass wire drawing with quantified uncertainties. International Journal for Uncertainty Quantification (accepted). 35
Thank you!
36