Estimated reading time: 4 minutes
If you missed our February NET+ AWS Strategic Call, you missed a lively discussion on one of the most pressing challenges facing research institutions today: how to design networking infrastructure that can handle massive data transfers without breaking the bank. AWS Solutions Architects Kevin Murakoshi and Nick Kniveton shared strategies that could save institutions thousands in unnecessary costs.
The Hidden Costs of Moving Research Data
The conversation quickly revealed how easily networking costs can get out of hand when supporting data-intensive research workloads. While compute often gets the spotlight in discussions about cloud costs, it's the data movement that can unexpectedly dominate budgets in research environments.
Research computing presents unique challenges: massive datasets transferred between compute resources, sporadic high-intensity processing periods, and collaborations that span multiple accounts, projects, and regions. Each of these characteristics creates potential cost traps.
Kevin walked through a compelling real-world example showing how research projects can unknowingly spend thousands of dollars monthly just on cross-availability zone data transfers—a sobering reminder that even small per-gigabyte costs add up quickly at research scale if you do not architect your workloads thoughtfully.
Strategic Options for Multi-VPC Research Environments
The AWS team examined three strategies with remarkably different cost implications:
VPC peering works beautifully for straightforward connections between two research environments, remaining the most cost-effective option with free data transfer within the same availability zone.
Transit Gateway shines as networking needs grow more complex. This hub-and-spoke model simplifies management, though it introduces data processing fees of $0.02/GB.
VPC Sharing emerged as particularly well-suited to the ephemeral, high-burst nature of research computing. This approach allows multiple AWS accounts to share a single VPC infrastructure.
VPC Sharing: A Game-Changer for Research Computing
Nick explained how VPC sharing aligns perfectly with the realities of research computing, generating significant interest during the session.
The separation of duties concept clearly resonated—network engineers maintain central control while researchers maintain autonomy over their workloads. This approach has the potential to transform current architectures at many institutions.
By sharing NAT gateways and other networking resources across multiple research projects, institutions can dramatically reduce duplicative costs. Early adopters have seen significant networking cost reductions while improving performance for their researchers.
Real-World Implementation Challenges
The discussion dug into practical implementation concerns including limitations (keep participant accounts under 100 per VPC), billing mechanics (VPC owners pay for infrastructure while participants pay for resource usage), and migration strategies.
The AWS team also addressed current limitations in tracking detailed data transfer costs. While AWS has received feature requests for improved cost attribution capabilities, they outlined practical workarounds for the present.
Community Knowledge Sharing
What made this call especially valuable was the rich exchange of real-world experiences from the community. The session highlighted examples of custom infrastructure-as-code tools that have streamlined VPC sharing implementation, and practical applications supporting multi-institution research collaborations.
Getting Support for Your Implementation
Need a strategic architecture review? Your AWS Solutions Architect can provide personalized guidance tailored to your specific research environment needs.
Ready for hands-on implementation help? The team offers "tech jams"—collaborative working sessions with AWS experts where you can tackle specific networking challenges together.
Looking for peer advice? The Internet2 NET+ AWS community provides ongoing forums where you can connect with colleagues who have already implemented these approaches.
Join Us Next Time
If this recap has you wishing you'd been part of the conversation, make sure you don't miss our next NET+ AWS Strategic Call in March. These monthly sessions bring together bright minds in higher education cloud computing to tackle common challenges and share innovative solutions.
Be sure to check out the other blog posts we've written. As always, feel free to send any feedback to tmanik[at]internet2[dot]edu.