Oregon State University’s legacy account management system presented significant challenges to meeting the growing needs of the institution, and providing access to anyone outside of our traditional population of students, employees, and campus associates. Our environment already contained Shibboleth and Grouper, and the goal of a common, higher education entity registry, was a significant factor in Oregon State becoming a TIER funding institution.
With the Campus Success Program, Oregon State implemented midPoint as our first standalone entity registry. Replacing decades-old business and technology processes is a large effort that was not fully achievable in the single-year CSP program, however, the midPoint deployment provides the technical foundation required to achieve these goals.
Oregon State’s original project scope was to implement midPoint as an entity registry and provisioning engine, and replace our existing account creation and management scripts. While the goal was to complete this work within the CSP timeframe, a lack of Identity & Access Management (IAM) resources required a change of scope. The revised scope for Oregon State was to implement midPoint, add existing identity records to the registry, and use midPoint’s provisioning engine to create our accounts in LDAP and Active Directory.
The effort to replace our existing legacy processes, serve external constituencies, and move to near real-time integrations will continue beyond the close of CSP.
TIER Feature Supported
- Oregon State’s IAM, Banner, and Infrastructure teams
- Keith Hazelton
- Warren Curry
- TIER CSP Banner-MidPoint Integration WG
- TIER API Working group
- TIER-midPoint slack channels
- TIER CSP schools
- midPoint mailing list.
Oregon State was a participant and significant contributor to the CSP Banner-midPoint Integration working group. By having IAM and Banner teams from multiple schools collaborate on the future of how IAM integrations should work, Oregon State gained valuable insight into our future direction for Banner data integrations. The work was facilitated by SMEs from the TIER API and Registry working group, who provided best practice architectural guidance.
Information Services (IS) builds and maintains a technology ecosystem at Oregon State that enables scholarship, learning, and community engagement in an environment where innovation and academic excellence thrive. Identity and Access Management is responsible for providing a robust and well-integrated infrastructure to identify people and allow access to systems, services and data. This includes both university-wide access rights as well as services to allow colleges, departments and groups to easily and securely create privileges.
There are several significant challenges presented by the current IAM system architecture. We have a collection of Perl scripts that are more than 15 years old which uses a custom table in our Banner database that serves as a registry. IAM lacks an ability to easily create accounts for individuals that do not exist in Banner, so we can only serve our traditional constituencies of employees, students, and campus associates. We are unable to adequately serve our 13,000 trained extension volunteers, the 9,000 people who participate in our Professional and Continuing Education (PACE) programs annually, parents of students, or the public at large.
The architecture of our current system is a scheduled batch process for account creation and management. The pace of business, and a campus expectation of moving at internet speed, requires us to support a near-real-time process for creation and updates.
Oregon State’s existing account management system consists of custom-engineered Perl scripts and batch jobs for provisioning and deprovisioning accounts. Batch jobs run every two hours to populate person information in a custom database table in Banner that serves as our registry. The scripts run every two hours to process the changes in the database table. These legacy scripts function as designed, but meeting the changing needs of our institution requires a complete replacement.
Oregon State’s vision for our IAM future will have identity data flowing from Banner and other future systems of record into midPoint and Grouper in real time. Identity information will be mastered in midPoint. Attribute-based access policy will be defined and managed in Grouper. Eligibility changes will be communicated from Grouper to midPoint, which will provision or deprovision accounts and services.
The opportunity to participate in the CSP provided us the motivation required to begin a modernization and major overhaul of our IAM infrastructure. This project will consist of multiple phases as we “replace the entire plane in mid-air.” The key TIER component for this project is midPoint – an identity registry and provisioning engine. midPoint’s registry will store our identity data, and its provisioning engine will manage our accounts. We will replace the Perl scripts, batch jobs, and the database table with midPoint.
The first phase of the project focuses on inserting midPoint into our account provisioning and deprovisioning workflow by replacing key parts of our existing script logic with calls to midPoint’s API. When we assign roles in midPoint, its provisioning engine will create accounts in LDAP and Active Directory. Removing roles in midPoint will deprovision accounts.
After the first phase, additional changes will be implemented in small steps. Phase two will focus on replacing individual functions of the existing Perl scripts with midPoint functionality, until we can retire the Perl scripts entirely. Following that, the goal is to make midPoint authoritative for more account attributes, allowing additional data to flow through midPoint instead of relying upon external scripts to modify accounts.
In future phases, we will use the Banner Ethos API to receive events in real-time from Banner. We will also connect Grouper to midPoint. When we break our dependency on our Banner custom registry, we can begin serving users outside our traditional constituencies. We will be able to have additional systems of record so that we no longer are tied to having a person record in Banner for every user.
Oregon State learned a significant amount about how to run and use midPoint during the course of the CSP. Through collaboration with other institutions, we worked toward a common set of Banner integration requirements and learned about the Banner Ethos API. This will facilitate our future work.
Replacing our legacy system is being completed in small steps. We are running our midPoint implementation alongside our legacy scripts, scheduled batch processes, and Banner registry table as we slowly migrate the functions. We completed the import of our existing identities into midPoint, and now co-create new identities in our Banner registry table and midPoint. We replaced the logic for provisioning, updating, and deprovisioning of LDAP and Active Directory accounts from the legacy perl scripts with API calls to midPoint’s provisioning functions.
The IAM team expanded our devOps knowledge and practices as part of the project. We used Terraform to build our midPoint infrastructure in our existing VM environment on-campus and Ansible to deploy and configure midPoint and midPoint’s repository database. This tooling allowed us to easily deploy test environments when reviewing the 3.7 to 3.8 upgrade, test our deployment process, and overall reduce the amount of time required for installation and maintenance.
Oregon State launched midPoint in production in a read-only mode in September 2018. Having monitored the actions and impacts on accounts for a month, we plan to complete the production implementation in October 2018 as the CSP program draws to a close.
Oregon State believed the project scope was aggressive for a year-long implementation, and we had not adequately accounted for staffing risk. Of the original three-person IAM technical team, only one remained with the team during the entire year-long project. For a quarter of the year, the IAM team had less than 1.0 FTE staffing.
The midPoint open source community is not as robust as other TIER software such as Shibboleth and Grouper. There is not much guidance available for the implementation of ideas, examples, or other community contributions. As it was new in the TIER package, the SMEs and other CSP schools were learning along with us. As midPoint is a complex piece of software, the effort for learning how it functions was significantly more than we had planned. The community would benefit from the development of a deployment guide for midPoint.
Since we did not originally plan on a purchase of a support contract for midPoint, we ran into additional challenges. Without an Evolveum support contract, there is only limited mailing list support, and we were unable to file bugs against the software or any connectors. Unlike the other TIER software, the developers are not part of the higher education community. It does limit their participation via mailing lists and other interactions without a support contract in place.
The loose interpretation of the LDAP standard by Active Directory proved challenging when integrating it with midPoint, which implements a fairly strict interpretation of the LDAP standard. Data cleanup was required in our AD environment to ensure accounts had appropriate object classes applied for all attributes on the account. We had to implement additional checks and auditing to ensure that external processes do not break the stricter implementation of object classes and attributes. We also had to make schema changes to our Oracle Directory Server Enterprise Edition LDAP service to accommodate midPoint’s (correct) enforcement of the LDAP standard.
Replacing a long-standing legacy system that “just works” because the bugs were worked out years ago is a daunting task. Focusing on replacing very small pieces at a time in an iterative fashion provides significant risk mitigation compared to a larger wholesale implementation.
Readiness for technology and process changes needs to exist within the broader organization. While IAM was interested in Banner Ethos integration and deploying midPoint into AWS, our organization was not fully ready for these changes. It is important for the organization to be ready for changes to interconnected systems, not just the individual project team.
https://github.com/OSU-IAM/ansible-midpoint Ansible playbook for midPoint
https://github.com/OSU-IAM/midpoint-config OSU’s midPoint configs
http://lists.evolveum.com/mailman/listinfo/midpoint midPoint mailing list
https://evolveum.com/services/training-and-certification/ midPoint training
The CSP program provided significant value to Oregon State. While an entity registry has been on our long-term roadmap, and was a primary factor in becoming a TIER funding institution, it was not previously slated it into our work plans. CSP provided the motivation to prioritize our work and focus on this foundational technology. Collaboration with other institutions working on similar issues and assistance from the SMEs was valuable to a small team working on new technology.
The implementation of midPoint provides us with a solid foundation to replace our legacy account management scripts. It also gives us the future flexibility to begin serving users outside of our traditional constituencies and better meet the needs of our campuses.
About Oregon State University
Oregon State is an international public research university located in Corvallis, Oregon. With 11 colleges, 15 Agricultural Experiment Stations, 35 county Extension offices, Hatfield Marine Science Center in Newport, and OSU Cascades in Bend, Oregon State has a presence in every one of Oregon’s 36 counties. The 4,700 students in our Ecampus extend this presence across the nation and around the world.
Oregon State is one of only two land, sea, space, and sun grant institutions in the U.S. and is the only university in Oregon to have earned both Carnegie Classifications for Highest Research Activity and Community Engagement. It is the state’s largest comprehensive public university, preeminent for both scholarly achievement and the direct impact of applied development, fulfilling the land-grant mission to serve the public good.