$10 million NSF grant funds next four years of multi-institutional project

Since it launched in 2015, Chameleon has enabled systems and networking innovations by providing thousands of computer scientists with the bare metal access they need to conceptualize, assemble, and test new cloud computing approaches. 

Under a new four-year, $10 million grant from the National Science Foundation (NSF), the cloud computing testbed will further broaden its scope, adding new features for reproducibility, IoT and networking experimentation, and GPU computation to its core mission. This multi-institutional initiative is led by the University of Chicago (UChicago) in collaboration with the Renaissance Computing Institute (RENCI), Texas Advanced Computing Center (TACC), and Northwestern University.

“Chameleon is a scientific instrument for computer science systems research,” said Kate Keahey, senior computer scientist at Argonne National Laboratory and the Consortium for Advanced Science and Engineering (CASE) of the University of Chicago, and principal investigator of the Chameleon project. “Astronomers have telescopes, biologists have microscopes, and computer scientists have Chameleon.”

In its first five years, Chameleon has attracted more than 4,000 users from over 100 institutions, working on more than 500 different research and education projects. Scientists have used the testbed to study power management, operating systems, virtualization, high performance computing, distributed computing, networking, security, machine learning, and more. Educators have used Chameleon for cloud computing courses, allowing college and high school students to build their own cloud and learn the inner workings of the technology. 

The upcoming phase of Chameleon will further develop work already begun such as the popular CHameleon Infrastructure (CHI) that provides enhanced capabilities with the open source OpenStack project. 

The team will also broaden connections to other mission-specific testbeds, which will allow experimenters to implement core contributions of testbeds beyond Chameleon into their work. For example, Chameleon will expand capabilities for connecting IoT technologies by integrating with testbeds such as SAGE.

RENCI’s contributions to Chameleon in the third phase of funding will support this cross-testbed capability by further enabling experimentation with advanced programmable networking devices and accelerators. The RENCI team will also develop new options for software-defined networking that will allow compatibility with FABRIC, a currently-developing “everywhere programmable” nationwide instrument with large amounts of compute and storage, interconnected by high speed, dedicated optical links. 

“The planned additions to Chameleon will allow academic researchers to experiment with advanced programmable networks in a large-scale cloud environment,” said Paul Ruth, assistant director of network research and infrastructure at RENCI and co-PI on the Chameleon project. “We are excited to extend Chameleon’s cloud experiments into RENCI’s FABRIC testbed, which will facilitate larger, more diverse networking experiments.” 

Finally, the Chameleon team will also add expanded tools for reproducible research, and they will add new hardware and storage resources at the project’s two primary sites, UChicago and TACC, as well as at a supplemental Northwestern University site.

“Chameleon is a great example of how shared infrastructure with over 4,000 users can save the academic community time and money while catalyzing new research results,” said Deepankar Medhi, program director in the Computer & Information Sciences & Engineering Directorate (CISE) at the National Science Foundation. “NSF is pleased to fund Chameleon for four more years in order to extend the platform with new capabilities, thus allowing researchers to conduct new lines of research and students to learn newer technologies.”

To learn more about the testbed or begin experimenting on it today, visit chameleoncloud.org.

iRODS users and consortium members will gather virtually from June 9-12

The worldwide iRODS user community will connect online this week for the 12th Annual iRODS User Group Meeting – three days of learning, sharing of use cases, and discussions of new capabilities that have been added to iRODS in the last year.

The virtual event, sponsored by the University of Arizona, Cloudian, and RENCI, will be a collection of live talks with Q&A. An audience of nearly 300 participants representing dozens of academic, government, and commercial institutions is expected to join.

“The annual iRODS User Group Meeting has always opened our eyes to the impact of iRODS worldwide, and this year’s meeting will be no different,” says Jason Coposky, Executive Director at iRODS. “Although we are moving to a virtual platform, we intend to provide a similar experience to years past by ensuring there are plenty of opportunities for networking, discussion, and collaboration.”

Meeting attendees will learn about new updates such as hard links, direct streaming, and policy composition, according to Coposky. On June 12, the last day of the meeting, the Consortium team will run an iRODS Troubleshooting session, where participants can receive one-on-one help with an existing or planned iRODS installation or integration.

Last month, iRODS Consortium and RENCI announced the release of iRODS 4.2.8. A notable addition within the release was a new C++ rule engine plugin that provides an iRODS system the ability to convey hard links to its users. An iRODS system stores a hard link when replicas of two different iRODS data objects with different logical paths share a common physical path on the same host. When this occurs, metadata is added to both logical data objects for bookkeeping.

This year’s update to the iRODS S3 plugin shares the design and engineering underway to provide direct streaming into and out of S3-compatible storage. This rewrite uses the new iRODS IOStreams library and in-memory buffering to make efficient multi-part transfers.

With the addition of a continuation code to the rule engine plugin framework, iRODS users are now able to configure multiple policies to be invoked for any given policy enforcement point. The policy developers now have the ability to separate the policy enforcement points from the policy itself. Given this new approach, multiple policies can be configured together, or composed, without the need to touch the code.

As always with the annual UGM, in addition to general software updates, users will offer presentations about their organizations’ deployments of iRODS. This year’s meeting will feature 23 talks from users around the world. Among the use cases and deployments to be featured are:

• SmartFarm data management, Agriculture Victoria. Data management challenges increase with large datasets generated with new sensing technologies. This requires the development of standardised, automated, on line, authenticated and verifiable standard processes for uploading data for storage and analytics on computing facilities. Agriculture Victoria undertakes research and development in animal and plant production, chemistry, spatial information, soil and water science. Working with iRODS, Agriculture Victoria are piloting new data management workflows of ‘SmartFarm’ data, and this talk will discuss lessons from small, medium, and high data Agriculture SmartFarm use cases using edge computing and collaborative data infrastructure and the flow on development of capability for AVR researchers.

• Data management in autonomous driving projects, Aptiv. Aptiv is a global technology company that develops safer, greener, and more connected solutions that enable the future of mobility. The company deployed iRODS in production around 1.5 year ago, together with the start of the development phase of a big project on autonomous driving. The researchers will share how iRODS has assisted in tracking and migrating data between partners and within engineering groups responsible for data collection, manual and automatical analysis.

• Building a national Research Data Management (RDM) infrastructure with iRODS in the Netherlands, SURF. In the Netherlands, many universities are looking at iRODS to support their researchers, as they recognize the powerful potential of the tool in two areas: support for secure cooperation, and support over the entire research data life cycle. SURF, a national organization providing IT support and infrastructure for universities, is now working closely together with six universities towards building a national RDM infrastructure based on iRODS. Researchers from SURF will share a case study for the use of iRODS, not for a specific research group, but for an entire nation to enhance the support of their researchers by working together on this iRODS based infrastructure.

• Keeping pace with science: The CyVerse Data Store in 2020 and the Future, CyVerse / University of Arizona. CyVerse, hosted at the University of Arizona, provides a national cyberinfrastructure for life science research as well as training scientists in using such high performance computing resources.This talk will describe the current features of the CyVerse Data Store and plans for its evolution. Since its inception in 2010, the Data Store has leveraged the power and versatility of iRODS by continually extending the functionality of CyVerse’s cyberinfrastructure. These features include project-specific storage, offsite replication, third-party service and application integrations, several data access methods, event stream publishing for indexing, and optimizations for accessing large sets of small files.

Registration for the Virtual iRODS UGM will remain open throughout the week. See the registration page for details.

About the iRODS Consortium

The iRODS Consortium is a membership organization that supports the development of the integrated Rule-Oriented Data System (iRODS), free open source software for data virtualization, data discovery, workflow automation, and secure collaboration. The iRODS Consortium provides a production-ready iRODS distribution and iRODS training, professional integration services, and support. The world’s top researchers in life sciences, geosciences, and information management use iRODS to control their data. Learn more at irods.org.

The iRODS Consortium is administered by founding member RENCI, a research institute for applications of cyberinfrastructure at the University of North Carolina at Chapel Hill. For more information about RENCI, visit renci.org.

The iRODS Consortium, the foundation that leads development and support of the integrated Rule-Oriented Data System (iRODS) data management software, welcomes OpenIO as its newest Consortium member.

The Renaissance Computing Institute (RENCI) at the University of North Carolina at Chapel Hill will collaborate on an $8.8 million, 3.5-year effort to make the volumes of data arising from cancer research more accessible, organized, and powerful. This contract was awarded by the Frederick National Laboratory for Cancer Research on behalf of the National Cancer Institute.