HPC, Big Data, and Data Science
PIRA: Performance Instrumentation Refinement Automation
<p>PIRA is a tool to automatically filter and focus Score-P's profiling to relevant program regions.
This involves both static, i.e., source-code feature, and dynamic, i.e., runtime information, analysis.
It uses the whole-program call-graph representation MetaCG for its analyses and has been used for automatic (a) hot-spot detection and refinement, (b) scalability analysis, (c) kernel identification, and (d) MPI load-imbalance detection.</p>
<p>In this talk, we present an overview of MetaCG and PIRA together with its analyses and a focus on the most recent addition of automatic (MPI) load-imbalance detection.
Our experiments on the SPEC CPU 2006 suite show that PIRA automatically constructs overview measurements with runtime overhead < 10%.
For the load-imbalance detection, our experiments on MPI-parallel LULESH and the Ice-sheet and Sea-level System Model~(ISSM) show that PIRA keeps the runtime overhead below 15%, while correctly identifying the existing load imbalances.</p>
<p>PIRA and MetaCG are available under BSD 3-clause license at https://github.com/tudasc/pira and https://github.com/tudasc/metacg.</p>
Additional information
| Type | devroom |
|---|
More sessions
| 2/5/22 |
<p>Working with Big Data means that we need tools to organise and understand the data. And you don’t have to be a developer to search, aggregate and visualise your data. Whether you need an affordable business analytics tool or you want to analyse log data in near real time, OpenSearch can help you. And all of it through a visual interface of OpenSearch Dashboards.</p> <p>After listening to this talk you’ll understand the basics of working with an OpenSearch cluster and different use cases ...
|
| 2/5/22 |
<p>In this talk, I will present Arcon, a Rust-native streaming runtime that integrates seamlessly with the Apache Arrow ecosystem. The Arcon philosophy is streaming first, similarly to systems such as Apache Flink and Timely Dataflow. However, unlike all existing systems, Arcon features great flexibility when it comes to its application state. Arcon's TSS query language allows extracting and operating on state snapshots consistently based on application-time constraints and interfacing with ...
|
| 2/5/22 |
<p>Any conversation about Big Data would be incomplete without talking about Apache Kafka and Apache Flink: the winning open source combination for high-volume streaming data pipelines.</p> <p>In this talk we'll explore how moving from long running batches to streaming data changes the game completely. We'll show how to build a streaming data pipeline, starting with Apache Kafka for storing and transmitting high throughput and low latency messages. Then we'll add Apache Flink, a distributed ...
|
| 2/5/22 |
<p>Why build #4 on the Green500 using OpenStack? It makes it easier to manage. Cambridge University started using OpenStack in 2015. Since mid 2020, all new hardware is controlled using OpenStack. Compute nodes, GPU nodes, Lustre nodes, Ceph nodes, almost everything. OpenStack allows large baremetal slurm clusters and dedicated TRE (trusted research environments) to share the same images. Is this a cloud native supercomputer?</p>
|
| 2/5/22 |
<p>This short talk will disect the container ecosystem for HPC in four segments and discusses what to look out for, what is already settled and how to navigate containers in 2022.</p>
|
| 2/5/22 |
<p>Optimizing CPU management improves cluster performance and security, but is daunting to almost everyone. CPU management may seem complex, but it can be explained in such a way that even your inner toddler will comprehend. With this talk, we will give a path to success.</p> <p>You may have a multi-socket node cluster where your AI/ML workloads care about the proximity of your CPUs to GPUs. You may be running scientific workloads where you want to pin in cores within containers instead of just ...
|
| 2/5/22 |
<p>Working with big data matrices is challenging, Kubernetes allows users to elastically scale, but can only have a pod as large as a node, which may not be large enough to fit the matrix in memory. While Kubernetes allows for other paradigms on top of it which allows pods to coordinate on individual jobs, setting them up and making them play nice with ML platforms is not straightforward. Using Apache Spark and Apache Mahout we can work with matrices of any dimension and distribute them across ...
|
