Cloud-Based System Architecture for the Federal Domain, originally published at CorregidorTech.com.
— — — — — — — —
This article describes a proposed system architecture for a cloud-based, web-centered enterprise-level system for use within the federal government domain.
This is written in the context of a notional scenario in which the architecture is being described to an interested system stakeholder, though one who may not be an expert in system infrastructure and design.
Things to note before we jump in:
- This is a bit of a work in progress. And represents a “jack-of-all-trades / master of none” approach. This article attempts to address many aspects of a complex system but only at superficial depth.
- The use-case that this architecture addresses is the consumption, standardization and storage of data, and the ability to make that data available to authorized end-users.
- Microsoft, including Azure, technologies are those that are mostly referred to here. For no reason other than personal familiarity. It’s not an endorsement.
- Nothing written about here is in any way protected or proprietary information. This is a generic overview of a notional enterprise architecture. And limited as that by the constraints of the author’s knowledge.
With that said, let’s see what we’re talking about:
Don’t worry if it’s unclear now — figuratively or visibly. Let’s address each of the system’s aspects below, starting with the upper-left quadrant:
Network Edge
Making use of the network edge for data intake is especially useful in support of the proposed Future of Federal Safety Oversight, where data, such as calibration measurements from within a regulated quality system can provide the government users real-time insight to inform rapid oversight decisions.
This system design takes advantage of Hardware-as-a-Service offered by the cloud service provider, like Azure’s Stack Edge to place computing power in proximity to sources of high-volume and/or complex data, such as major manufacturing facilities, to perform immediate large-scale processing before transfer to the cloud for persistent storage.
Services such as Azure Edge Zones allow the system to connect with 5G and LTE networks to further support IoT integration.
External Users
Non-government users can access system services and enter, view, and consume data via a public portal available on the internet.
Okta, or another third-party identity management provider like it, manages the identities of the portal’s authenticated non-government (“external”) users; allows Single Sign-On if multiple apps exist; multi-factor, behavior-based authentication; and an interface for the federal users to administer these services.
The identity management service may enable authentication via various protocol, including OAuth, Proof Key for Code Exchange, and Open ID Connect.
While registering for access to the system, external users may be required to digitally confirm their identity. The LexisNexis Risk Defense Platform provides a service that accomplishes this by asking the user questions to which only she/he should know the answers.
The client-side application utilizes a modern JavaScript library or framework, such as React, Angular, or Vue, to enable an interface that empowers significant interaction with the user, including the temporary storage of data to minimize the need for perpetual data calls back and forth to/from the server.
In cases where it may be beneficial to render web assets on the server rather than in the user’s browser, a framework that targets server-side rendering in a modern way, such as .NET Blazor, may be optimal.
We take advantage of JavaScript serviceworkers to enable offline functionality.
The system may require dedicated mobile apps. Development options include the use of NativeScript or React Native that may allow the mobile app to implement code — or at least learning — from the Angular, Vue, or React web application. Mobile-native apps can be written in Java or Swift for Android and iOS devices, respectively.
The client-side applications and network edge devices can benefit in terms of bandwidth and responsiveness as well as security and availability by use of techniques traditional and ubiquitous in modern web-based systems, including content delivery networks, which can be obtained as a service from the systems’ cloud provider.
In-memory caching systems such as key/value stores, like Memcached and Redis, can also enhance performance of client applications and devices.
Microservices
Along with the data persistence layer, the microservice infrastructure is central in leveraging the cloud in this modern system design. In the diagram, the set of microservices is depicted twice as both applications, the external users’ portal and the internal application for federal users, have microservice architectures.
Benefits of microservices include the ability to easily stand up redundant services to ensure constant availability of critical functionalities and the ability to quickly prototype and modify services in support of stakeholder needs.
Each microservice is developed in and deployed to a container (Docker, LXC). The containers deployed to the cloud may be managed by a container orchestrator like Kubernetes. Alternatively, microservices could be deployed to virtual machines. Containers provide the advantage of allowing developers to build to an environment that matches the one to which the microservice will be deployed.
An event-driven approach drives the microservice design. One strategy may be Command Query Responsibility Segregation (CQRS). Well-planned design patterns in microservice code ensure that events are shared among microservices and data state is synchronized. Service meshes, and protocols such as gRPC, may also be utilized to share requests and data among microservices.
Each microservice functionality is presented to the user via the client application. Each of these functionalities is supported (in the case of .NET) via the Web API controller which creates a REST data interface to which the client/mobile application(s) can integrate. Smartly designed action methods within the controller can consume, manipulate via business rules, and expose the data for use by the client application(s). The action methods within the controllers query the data source via an object relational mapping tool. An example, when using .NET with relational (SQL) data is Entity Framework.
Cloud-hosted “serverless” functions can, in some cases, support the microservices without the need to write and deploy an entire backend solution. Among benefits, serverless functions support upward scalability by easing the enhancement of services.
Each microservice may be supported by its own small dedicated source of virtualized/replicated data.
Cloud Storage
Utilizing the cloud storage provider, we have a data lake of raw data to for real time data needs supporting, for one example, dashboards. Separately, data is stored long term in a data warehouse which can serve enterprise needs, including as the source of truth for each of the microservices’ smaller stores, and for data engineering and business analytics.
A primary goal of this system infrastructure is to intake, consolidate, and make available data for both immediate decision-making and long-term strategic planning. Stored data should be accessible by authorized downstream clients via various protocols, including REST, and GraphQL and in common data formats to include JSON, binary, and XML. System users can integrate the business intelligence / machine learning tools best suited for their business process needs by consuming the system’s shared data.
The system design will be best prepared to provide its required functionalities if the architect understands well the business processes that it supports. The data classes in the backend solution (in this case, via .NET) and the database schemas and records should map to business process entities per a domain driven design approach. Rigorous documentation is essential to system maintenance and future enhancement, and should feature data models representing the physical, logical, and conceptual forms. And system design documents that describe how data persistence aspects map to features in the user applications.
Internal Users
Like Okta for external users, a cloud-based service helps manage identities of the system’s government “internal” users. The Azure Active Directory provides this service with the Microsoft technology stack. Though, Okta among other services, works for this purpose as well.
Government users should be able to authenticate access to the system via their Personal Identity Verification (ID card). Our system doesn’t define its own custom solution. Some federal agencies provide their own per the Federal Identity, Credential, and Access Management (FICAM) program. This system design assumes PIV card authentication via a Security Assertion Markup Language (SAML) integration with the FICAM service.
Common
Aspects common among development and management of the source code at most layers of the system include:
- A shared library of components that can be inherited in user interfaces throughout the system, promoting uniform, branded appearance and behavior.
- The use of open source collections of software libraries such as NuGet and NPM to import code and also to upload code assets shared amongst the development team.
- Source code can be stored in the cloud via GitHub and Bitbucket, from which the code may be shared, and branched from easily by the development team. Cloud-hosted source code repositories support modern version control and code review practices, and automated continuous integration / continuous delivery (CI/CD) pipelines.
- Code quality can be enhanced via a test-driven approach or by ensuring an adequate level of automated code coverage via unit tests (required passing can be incorporated into the CI/CD workflow), and other forms of quality assurance such as browser automation, regression, and user acceptance testing.
- In the author’s experience, Agile has proven successful as a software development project management methodology — in one specific example described HERE. Implementation stories may be assigned and tracked via an online tool such as Jira.
Thank you
This is by no means an exhaustive definition of enterprise system design. Maybe it’s volume 1! Perhaps this article may have been most effective, truly, as a moderate research project for this Medium contributor — who is grateful that you chose to read along. Thank you for reading!
