The 2018 Technical Congress was all about the Propeller-Heads. The opening plenary invoked the propeller-head moniker – an alternate term for computer geeks and techies – to remind the audience that the Technical Congress is the arena of the technical and subject matter experts who make up the Ed-Fi community. The purpose of this gathering of geeks and techies is to create a space for improving the Ed-Fi Data Standard and related technology.
The sixteen sessions and three plenaries offered a deep dive into exciting developments and on-going challenges in “seamlessly and securely connectingeducational data systems.” This summary attempts to capture the highlights, and particularly focuses on the outcomes and next actions from this meeting.
Upgrade to Ed-Fi Suite 3
The release of Ed-Fi Suite 3 components, both Data Standard v3.0 and ODS/API v3.0, are currently slated for May, per the roadmap. Multiple sessions focused on what is changing in the Data Standard and ODS / API, and what the transition will look like for vendors and education agencies alike. Some highlights include:
- Most importantly, the ODS/API v3.0 supports the new and changed resources described in the Data Standard v3.0
- Decoupling of Ed-Fi Core and Extensions in JSON data and API routing conventions.
- Changes to the Swagger-based API documentation.
- Changes to “2-legged” client credentials grant OAuth from “2.5-legged” model.
Discussion around mutual support during the upgrade process, especially around state extensions to the API, resulted in calls for a new Special Interest Group (SIG) to meet regularly to help resolve issues and share resources as the Suite 3 goes into production. Whether you are ready to start the upgrade in 2018 or just beginning to evaluate, state agencies and vendors are invited to join the Data Exchange and Extensibility in Ed-Fi Suite 3 SIG. If interested, please contact Sayee Srinivasan (sayee.srinivasan@ed-fi.org).
Connecting the Dots to the Ed-Fi Roadmap
Several conversations at the Technical Congress were motivated by concerns and field issues that already have a presence on the Ed-Fi roadmap. Below we attempt to “connect the dots” to those items. We welcome ongoing discussion (via Slack or Tracker, or participation in the governance process) on any of these items.
- Ed-Fi Dashboards and Suite 3: There was some confusion over the ongoing support for the Ed-Fi Dashboards and we want to clarify that an Ed-Fi Dashboards v2.1 release is on the roadmap for release in September 2018. That release will provide Dashboard compatibility with the Ed-Fi ODS / API v3.1, plus any minor bug fixes.
- Ongoing Development of Assessment and Finance Data Exchange Standards: Under the new governance process, Work Groups are being convened on these two topics. The Work Groups will coordinate community standardization of data and API specifications for their respective domains. Recommendations from the Work Groups will be sent to the new Ed-Fi Governance Advisory Team for review and coordination of approvals for related project work.
- Data Out at Scale via Secure REST API. Several community members raised this topic (it was in fact mentioned 3 times in the opening Darts and Laurels plenary!). The root issue is how to use the current “granular” Ed-Fi API to securely access large amounts of data. Too often in our industry, such access is accomplished by direct database connections and other practices not clearly safe or secure, so we are delighted to see API access as preferred. But synchronizing large amounts of data via a REST API over a highly normalized data model can be inefficient. In this area, we have a couple of notes:
- The new Change Events API is on the roadmap for ODS / API v3.1 (Q3 of 2018). This API allows for querying that only includes changes to existing API resources.
- Bulk Movement of Data via JSON and new API endpoints is also being evaluated, and the Alliance is forming a Special Interest Group to evaluate proposed designs for that. If interested, please contact Stephen Fuqua (stephen.fuqua@ed-fi.org)
- API Validations: While the Ed-Fi ODS/API validates that POST request payloads have required data, there is no mechanism for adding custom business rules for more complex validations. Some community members have asked about creating a validation framework that acts BEFORE the data lands in the API. The Alliance and the Technical Advisory Group took a look at this question in 2017 (see summary and meeting notes here) and concluded that such an approach ran the risk of fragmenting the vendor experience with Ed-Fi APIs and would therefore not be supported. We invite ongoing feedback on this decision.
- Temporal Data: The Temporal ODS project has been exploring multiple options for capturing and storing multi-year data. A Temporal ODS Early Adopters Program (to be run as a Special Interest Group) was announced during the Technical Congress. Organizations interested in participating need to apply. See details here.
- Installer Options: The Azure and AWS deployment configurations will continue to help those who want to install their Ed-Fi ODS/API in the cloud. But the need for more help installing on-premise was heard loud and clear. Work is already underway to create a simple Windows Installer, and to provide sample Docker build scripts, to be published to the Ed-Fi Exchange. Look for these in the Exchange this summer.
- Data In From Smaller and “Informal” Systems. Smaller software products and Excel spreadsheets are part of the reality of the K12 sector today, and those can often hold valuable data. When a system can’t connect to the ODS via the REST API, you’re usually left with ETL of CSV files. Loading these into the ODS is not a trivial matter due to the highly normalized relational database. Improved support for this is planned on the roadmap via the “Long Tail of Data” Solution.
- Analytics Out from a Highly Normalized Data Model. Ed-Fi defaults to highly normalized, granular data because such data can work for everyone. But such a model also makes calculations (e.g. Average Daily Attendance) very difficult. We invite community members to watch for and send feedback on the “Missing Metrics Layer” (de-normalized data for reporting and analytics) project on the roadmap.
New and Open Issues
Two new topics with no current roadmap presence were raised and will need to be addressed by the Alliance and community.
- Data standards governance for state education agencies: State education agencies need to extend the data standard for proprietary purposes, and yet those purposes often overlap. Without coordination, fragmentation undermines vendor ability to serve multiple markets and improve product quality. What forum would enable agency collaboration for developing and sharing common data models? The provisional next step is to see if the Data Exchange and Extensibility in Ed-Fi Suite 3 SIG can help suggest and evaluate options.
- Certification decomposition: The Ed-Fi certification process covers a large set of domain concepts, only some of which are relevant for each agency. Some vendors who focus on specific markets may be able to meet a subset of the SIS certification requirements but not the entire ODS/API definition. Could the requirements be decomposed into a few logical groupings? The challenge with breaking apart the certification is in right-sizing the covered domain and continuing to maintain a service mark that adequately communicates about quality and breadth of compatibility. The Ed-Fi Alliance intends to bring this topic to the Governance Advisory Team (GAT).
Wrap Up
The Technical Congress sessions were interactive, productive, and educational. The feedback has been very positive – and not just for those who appreciated escaping the last gasps of winter back home for the spring warmth of Scottsdale. We owe kudos and thanks to our hosts – the Arizona Department of Education – and to all presenters, but perhaps most of all to all of you who shared your laurels, your darts, and your on-going concerns.
Slide decks from the sessions and plenaries, as well as working notes from most sessions, are available in 2018 Technical Congress on Tech Docs. The #technical-congress channel on Slack remains open and is a great way for participants to reconnect on issues.
