Integrated Parallel Simulation and Visualization

People involved: Preeti Malakar

The emergence of the exascale era necessitates development of new techniques to efficiently perform high-performance scientific simulations, online data analysis and on-the-fly visualization. Critical applications like cyclone tracking and earthquake modeling require high-fidelity and high-performance simulations involving large-scale computations and generate huge amounts of data. Faster simulations and simultaneous online data analysis and visualization enable scientists provide real-time guidance to policy makers. We developed a set of techniques for efficient high-fidelity simulations, online data analysis and visualization in environments with varying resource configurations. We propose parallel execution of nested simulations. A novel combination of interpolation-based performance prediction, Huffman tree-based rectangular partitioning of the 2D process grid and topology-aware mapping results in up to 33% gain over the default strategy in weather models. Additionally, a novel tree reorganization strategy based on hierarchical diffusion minimizes data redistribution cost while reallocating processors in the case of dynamic regions of interest. Our method is able to reduce the redistribution time by 25% over a simple partition from scratch method.



Huffman tree based processor allocation

It is important to consider resource constraints like I/O bandwidth, disk space and network bandwidth for continuous simulation and smooth visualization. Formulating the problem as an optimization problem can determine optimal execution parameters for enabling smooth simulation and visualization. This approach proves beneficial for resource-constrained environments, whereas a naive greedy strategy leads to stalling and disk overflow. Our optimization method provides about 30% higher simulation rate and consumes about 25-50% lesser storage space than a naive greedy approach.



Adaptive framework which selects optimal parameters for simultaneous simulations and visualization

We have developed algorithms to minimize the lag between the time when the simulation produces an output frame and the time when the frame is visualized. It is important to reduce the lag so that the scientists can get on-the-fly view of the simulation, and concurrently visualize important events in the simulation. We present most-recent, auto-clustering and adaptive algorithms for reducing lag. The lag-reduction algorithms adapt to the available resource parameters and the number of pending frames to be sent to the visualization site by transferring a representative subset of frames. Our adaptive algorithm reduces lag by 72% and provides 37% larger representativeness than the most-recent for slow networks.



Temporal clustering of frames using modified kmeans, k is found by analyzing the rms distance between consecutive frames

Publications

1. Preeti Malakar, Vijay Natarajan, Sathish S. Vadhiyar and Ravi S. Nanjundiah.
   A diffusion-based processor reallocation strategy for tracking multiple dynamically varying weather phenomena.
   
2. Preeti Malakar.
   Integrated parallelization of computation and visualization for large-scale weather applications.
   
3. Preeti Malakar, Thomas George, Sameer Kumar, Rashmi Mittal, Vijay Natarajan, Yogish Sabharwal, Vaibhav Saxena and Sathish S. Vadhiyar.
   A divide and conquer strategy for scaling weather simulations with multiple regions of interest.
   
4. Preeti Malakar, Vijay Natarajan and Sathish Vadhiyar.
   Integrated parallelization of computations and visualization for large-scale applications.
   
5. Preeti Malakar, Vijay Natarajan, and Sathish Vadhiyar.
   INST: An integrated steering framework for critical weather applications.
   
6. Preeti Malakar, Vijay Natarajan, and Sathish Vadhiyar.
   An adaptive framework for simulation and online remote visualization of critical climate applications in resource-constrained environments.
   
7. Preeti Malakar, Vijay Natarajan, and Sathish S. Vadhiyar.
   A coupled framework for parameter simulation and visualization.
   
8. Preeti Malakar, Vijay Natarajan, Sathish S. Vadhiyar, and Ravi S. Nanjundiah.
   An integrated simulation and visualization framework for tracking cyclone Aila.
   

Software

1. InST
   An Integrated Steering Framework for Critical Climate Applications.

Contact

malakar [dot] preeti [at] gmail [dot] com