Past Event – 2022 Program

To make the most of the extremely high-bandwidth network capacity that is available within (and out of) Australia for the research and education sector; a uniform, parallel and secure data transfer and management service is needed. Too often, Australian researchers have been resorting to using portable storage devices to move data around as transfers fail. A service to support data movement, leveraging the network was needed.

During 2021/22, AARNet piloted Globus with the Research and Education sector. Globus, providing fast, parallel and secure file transfers and data management; proved to be well equipped to support researchers in meeting their transfer and data movement needs. The deployment of Globus across Australia, would also standardise transfer endpoints across the research and education sector’s facilities.

We will present the early results and learnings of the utilisation of the service, along with future-plans for rollout across Australia to support growing data needs of institutional researchers. Further, we will highlight the opportunities the sector has identified through the utilisation of this service.

I will discuss a variety of examples that showcase how AI, scientific data infrastructure and supercomputing may be combined to address computational grand challenges in big data physics experiments.

Abstract coming soon.

The Earth System Grid Federation (ESGF) is a global peer-to-peer network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. After recently providing data from a variety of climate change scenario projections to inform the United Nations Intergovernmental Panel on Climate Change (IPCC) Sixth Assessment Report (AR6), ESGF will be re-architected to improve its reliability, scalability, and usability for the next generation of high resolution global Earth system model outputs from dozens of international modeling centers. Building on Globus technology, ESGF2 will move data as needed among US high performance computing centers and across simulation and analysis platforms to enhance the productivity of Earth system scientists.

AbbVie, a rapidly growing global pharmaceutical company has several research centers in US and Europe. The company relies more and more on in silico development for drug discovery and process development, with heavy use of HPC resources. Compute and storage resources are consolidated in the major processing centers in the US due to ever increasing requirements for power and cooling. Data is generated at the company research centers and needs to be transferred for processing to the computational centers and public cloud solutions. The company centers are inter-connected with corporate intranet leveraging high speed SDWAN solutions. Numerous firewalls are deployed separating the corporate intranet from the Internet and public cloud. No incoming connections are allowed to internal resources from the Internet per corporate policies.

Ever growing data sets require new ways of transferring data. After extensive testing of in-house data transfer applications and native operating systems’ tools, Globus Connect Personal demonstrated high data transfer rates. Globus Connect Personal didn’t require incoming connections from the internet, which complies with corporate policies. The company has deployed a series of Globus Connect Personal endpoints across the globe, leveraging existing virtual HCI infrastructure to allow data transfers over corporate high speed SDWAN solution. Globus Connect Personal Endpoints provided high data transfers rates, easy access to users’ data and reduced data friction for individual users and groups.

The GenePattern platform for integrative genomics has supported the analysis needs of the worldwide genomics community since 2004, with its main cloud-based server providing thousands of analyses per week for a user base of over 80,000. The web-based gateway provides hundreds of analysis methods in an accessible format that can be used without the need for programming and provides extensive capabilities for the creation of workflows and the reproducibility of analyses. In 2017, we released GenePattern Notebook, a Jupyter Notebook-based interface allowing scientists to embed GenePattern-hosted analyses within notebooks and providing additional functionality to support genomics workflows for scientists at all levels of programming expertise.

As new genomic technologies such as single-cell RNA-seq and spatial transcriptomics become more prevalent, the size of datasets challenges the architecture of web-based gateway applications. We collaborated with Globus to incorporate Globus data transfer into our GenePattern application to provide robust data transfer of large datasets seamlessly and intuitively within the gateway user interface. We will discuss the science that our Globus integration enables and the challenges of adapting Globus data transfer to a multi-user web application architecture.

The cost of collaborative cloud storage is quickly growing beyond the reach of many institutions used to the halcyon days of free or low-cost unlimited storage from Google, Box and others. Those days are now gone, replaced by careful examination of storage use, drafting of data lifecycle and storage guidelines, as well as the development of strategies for distributing the data to more appropriate platforms where, in some cases, the costs can be more directly borne by the owners of the data. All of this planning is contingent upon being able to move the data from one location to the other both affordably and manageably. Could Globus be the answer to those challenges?

The speakers on this panel come from a range of public and private R1 institutions and include storage admins, research technologists, Globus admins and those who must bring them together. At this point the challenges outnumber the solutions, so expect a lively debate around questions disinclined to easy answers.

The ESnet Science DMZ model has been widely recognized as the gold standard in network design to support data intensive science use cases. This model, widely deployed throughout the R&E community, has adapted to changes over the previous 10 years but still features a core set of design principles to ensure friction free data movement, while enabling a high degree of security and flexibility.

This panel will introduce several different deployment models, featuring members of the Globus Community, and will touch on issues of performance, security, and onboarding users. Bring your questions to ask the panelists about how they have convinced their leadership of the needs, costs, and outcomes in adopting this approach.

Data mobility is critical to most scientific workflows, and Globus helps to enable this for a large section of the R&E community. Is it possible to determine if your site or facility is performing at the peak capabilities of the software, hardware, and network you have available? The Data Mobility Exhibition provides the means to run simple tests, evaluate expectations and seek assistance for making improvements.

The de facto standard for research data management is applying the Software-as-a-Service model to research computing. Over the past two years the Globus Labs team has developed funcX, a service that facilitates function execution across a federated ecosystem of compute endpoints spanning personal laptops through supercomputers. Having proven the model in select research projects, we would like the community's input as we work towards a generally available product.

We will provide an overview of the funcX service and solicit feedback from research computing practitioners on design, implementation, deployment and management to ensure the product effectively addresses the majority of use cases. This session is a unique opportunity for you to help shape upcoming product releases and will be optimized for in-person participation.