Logistics

• 08 December 2022
• Ashish Pandit (UCSD and Working-Group convener)
• Nick Austin (Kansas State University)
• Abdul Qureshi (Kansas State University)
• John Gunvaldson (UCSD)
• Matthew Hailstone (Brigham Young University)
• Swarnim Ranjitkar (UCLA)
• jeff kennedy (The University of Auckland)

Discussion

Business Value of APIs

The rough notes that follow reflect aspects of the discussion, and are deliberately not attributed to individual contributors.

• Various drivers considered up front:
  • technical drivers: developers wanting to experiment with and use APIs, but with high variability of style and quality and very little meaningful governance.
  • project needs: particularly for large projects like ERP refurbishment there are often strong demands for APIs to support integration.
  • student experience: seeking business-outcome drivers of having APIs that help improve student experience (e.g., in and around the student support call centre)
  • wider economics: having a University API product and the serendipity aspect of creating asset that people might hopefully find useful in the future.
• Largely driven by project needs. Are there two "sets" of APIs at play here: your business capability APIs
• As soon as you expose an API for business needs such as feeding a particularly application there will straightaway be additional demands for using that same API for getting access to the underlying data, but beware of API designs that don't support both transactional data access and also batch-based data access.
• CIO mandate (in the style of Jeff Bezos!) that there be more APIs.  There were lots of screens using a legacy data-access framework, and there was a deliberate move to refactor those screens to use RESTful data access patterns (up to four-hundred RESTful APIs created).  Migrated from SOASoft API Management tool across to WSO2 and moved access-management from custom access keys to OAuth and relying parties: has moved subsequently to Tyk with Hydra behind it as an open-source OAuth server.  There is now a University API project that is capable of retrieving data about primary resources (e.g., person and student), and moving now towards a more domain-focused approach to organizing and managing the API fleet.  Quite a lot of work involved in identifying all the various clients accessing data in various ways (e.g., through direct database connections) to a standard API-based approach.  Alongside all of this is an Okta implementation, and getting claims and SAML assertions set up in here (¿potentially bridging to OAuth?) is also underway.  Strong focus on reading data, not on updating the data, for these APIs.
• Master Data Management in the context of APIs?  If your APIs are connected directly to the system of record | source of truth then that is one approach (for the in-scope resources) to master-data-management.  For example, data about a student can come from various sources (e.g., residency and tuition and enrollment and housing...), so what is considered here to be the master version of the data?  How is that kind of situation reconciled?  In some cases, these are just separate sources of separate data (e.g., contact details for specific purposes that are managed in local applications related to those specific purposes: the delivery address for a diploma; the email address for work), but in extreme cases there could be "broken" MDM for things like unique identifiers for students (is it an ID Number or a Username or a Badge ID or an Email Address?) and how do we traverse/match across those various identifiers?  This is partly a data-management issue, rather than specifically an API issue, but there are questions about vocabulary?
• There is a difficult job for data architects involved in getting just-enough consistency across domains = common naming, and common data types, and common enumerations, and moving to something like a pseudo-canonical high-level data model across all of the APIs (at least for APIs that need to have that layer of consistency for consumability: those higher up than the "system" layer).
• Some discussion about "layers", whether the Gartner inner-xor-outer of the MuleSoft system-process-experience arrangement (see https://www.mulesoft.com/resources/api/types-of-apis for details).
• Where a SaaS application is providing APIs that we want to expose to our users, what are the patterns used to abstract, govern, throttle, protect those APIs for internal users (and to harmonize access-management and other aspects of the API behavior).  ZapThink had the architectural rule that "clients must never be permitted to establish direct connections with endpoints" — there must always be something interspersed between those two actors.
• What is the ultimate goal of your API program — is it to enable any and every user to access programmatically all of the data and functionality they are allowed to access through the screens of web applications?  What are the trade-offs involved here?
• Systems like Canvas and ServiceNow have full API coverage provided, but is it desirable to make these available (or close them off) from your users?
• The concept "back-end-for-front-end", BFF (e.g., https://learn.microsoft.com/en-us/azure/architecture/patterns/backends-for-frontends and https://blog.bitsrc.io/bff-pattern-backend-for-frontend-an-introduction-e4fa965128bf) comes into play here too, being able to limit and provide just the few fields that a particular caller might need.
• GraphQL does shift some burden onto the calling client, and there is some tradeoff there between the expertise and responsibility demands and the flexibility and convenience affordances.  The use of strongly-cast schema and the use of authorization techniques such as the directive resolvers are heavily recommended.
• It is crucial to establish strong governance around schema design and lifecycle management!

Data Management and Integration Prelude

Kansas State University

• three different ERP systems (Student and HR are PeopleSoft; Oracle eBS for Finance)
• also some long-standing Software AG middleware
• lots of data!
• starting a Snowflake data warehouse implementation
• the integrations team and the ERP team haven't worked together as closely as they could
• great opportunities for APIs here
• Informatica IICS also being implemented here
• AWS shop, so using Lambda functions on MQ for messaging out of the homegrown IAM

UCSD

• broadly shifting from batch processing to near-real-time and streaming architectures for their integrations
• Informatica IICS is involved here as a platform
• Apache NiFi
• AirFlow is a big part of this too = https://airflow.apache.org/
• streaming with Kafka
• the introduction of open-source elements around the IICS was driven by a desire to move away from the generic cloud connector and also because the API Manager functionality in IICS at the initial stages was still maturing