Oats〜i
I've been an active web developer for about five years now. Back in the end of 2019 when I officially started doing web development, I was surrounded by a vast ecosystem of web development tools that I could leverage to create websites for clients and any personal project I had.
But I found myself going the do-it-yourself route. Not because am accustomed to pain and head-scratching moments, but rather because I wanted to learn web development from its basics, and not jump right into a framework and build my knowledge from there.
Besides, that's what most experienced web developers advise. Learn HTML, CSS, and Vanilla JavaScript, and anything else on top of that will be a breeze in the park (sort of).
Well, here we are, five years later and somehow, I ended up making a web framework of my own. What started out as a simple learning exercise of how the web and web APIs work ended up becoming a full-blown project with countless head scratching moments, disappointments, and eureka.
Introducing Oats~i, the open web framework that also, takes you back to the basics. Oats~i provides a structure that allows you to create web apps using HTML, CSS, and Vanilla JavaScript, with powerful extensibility, server-side rendering, consent-based routing, reactivity through a data manager, view manager, and hooks, a fragment-based view system, view panels to support extra layouts, popups, and custom views on top of fragments, support for "native" web browsing features such as params, queries, and targets, pagination, code splitting, and lazy loading for JavaScript and view bundles.
All these come natively with the framework, running on top of Webpack as the preferred module bundler.
Oats~i doesn't care about your server environment because it's a purely client-side based system. There's no running JS on the server, so no extra special server setup is needed to deploy an Oats~i app.
But before we get into the details, where is it running now?
Here: https://www.vertesolutions.com
That site is a production site for a client dealing in eco-consultancy and eco-business. The business is called Verte Environmental Solutions, so if you find me referencing "Verte's website," that's the site am referring to.
Oats~i currently runs in no other place.
EDIT: The source code is also public. https://github.com/Oats-i/Oats-i
Before getting to this point, I've been developing the framework on the site's admin panel (which is custom made), testing, updating, and adding new features over the years. So, if you miss some of what I'll talk about here on the client site, be sure they're running on the admin.
Also, what I'm introducing is a rather fleshed-out piece of framework, with a couple of features to talk about. So be warned, this is going to be a long piece. And I've tried to edit it as much as I can to make it shorter. However, I've touched the gist of it all with this intro. The rest of the piece, is a scrape of the surface before we start getting deeper in future posts.
So, enough of this intro.
Let's get a little bit deeper into Oats~i, as I've built it so far, and look at some of what it offers out of the box and what am planning with it moving forward.
Edit: You love videos instead? Watch this intro instead on our YouTube channel: https://youtu.be/6E_Q25YfQYc?si=U4GjREyvERIP72i-
An Open Web Framework
By calling Oats~i an open web framework, I mean that Oats~i is a simple yet extensible framework whose code can be written in simple HTML and Vanilla JavaScript, with CSS of course being the de-facto styling tool. From this simple setup, you can add your own, third-party or custom templating engines, css libraries, and other tools, as long as Webpack allows it and you can configure it.
Fragment-Based System
Oats~i works through a build system that spawns up fragments as "components" or core pieces of the web app. Take these simple views for instance:
Both images have the "root view" of the app, which is the main view of the app that the user will always see. Then there are the other views (fragments) will be rendered dynamically within.
The root view can contain the primary navigation links or buttons, and other views that the user will always see on the app, and will typically not change.
The rest of the views inside the root view will change, and this will be based on fragments being loaded and offloaded off the app, based on the user's routing. Fragments go through a build process that primarily gets the view to be rendered, places it in the targeted parent node, and then allows you to wire the rest of your app and business logic.
An Oats~i build process will typically trigger the following core methods in your fragment:
//gets the view for your fragment
async initializeView(cb){
}
//triggers after the fragment view has been attached to the DOM
onUIBind(serverSideRendered){
}
//triggers after the fragment has completed building and anytime its rebuilt for the same route
async onQueryParamsDataUpdate(changedParams, data, savedState, routeParams, isServerSide){
}
And that's basically it.
With such a skeletal structure you have a couple of flexibilities, such as:
Rendering Simple HTML or Loading Complex Views Using Templating Engines (Of Your Choice)
The first method you override (initializeView()) can be completed like this:
async initializeView(cb){
const viewAsString = `<p class="text">My view</p>`;
this.onViewInitSuccess(viewAsString, cb);
}
We get our view as a HTML string and pass it to the internal method (onViewInitSuccess()) that also takes the callback that is passed to the original method.
Calling onViewInitSuccess() triggers the build process to continue to the next steps.
Writing HTML as a string within JS is simple and Oats~i allows for it but, it can often get problematic. However, instead of building a new syntax or system for writing views for Oats~i, Oats~i allows you to plug in whatever templating engine works best for your use case, wire it up in your webpack config, and let it work its magic.
For Verte's case, I use handlebars, combined with handlebars-loader to write separate view files in hbs format and simply require them in my code.
So, instead of
const viewAsString = `<p class="text">My view</p>`;
My views are now provided as:
const viewAsString = require("./relative/path/to/view.hbs")(myTemplatingData);
If I want to use ejs instead, for instance, I just have to update my webpack config and use the right import syntax for that use case.
Oats~i only cares that the view passed to it is a HTML string.
Network-Sourcing Views
Oats~i goes as far as allowing you to source your views over network. That's partially why the async exists on the initializeView() method.
Oats~i also expects you to possibly make network calls at this stage, either for a complete view based on user type or other factors, or get templating data based on your view and business logic.
What you do here is totally dependent on your business and technical reasons.
**NOTE: **There's a good reason why the build system doesn't wait for promises at the build stage to resolve using await or then(), but instead uses a callback passed to the relevant methods. That will be clear when we'll be diving deep into how Oats~i works, in a later piece.
Vanilla JavaScript or a Compatible JS Library for App or Business Logic
Oats~i code is in vanilla JavaScript, the "native" language web browsers understand. However, there are a few flexibilities that you can have when writing your business logic.
For instance, you can port jQuery in your project, for whatever reason, and use it to write part of your logic. I actually did this a long while back, even before Oats~i had been built to its current state, to write about five lines of code for smooth scrolling effects in Verte's website. (TLDR, I was lazy to think beyond Stack Overflow, lol).
You can theoretically use Oats~i in a TypeScript environment, but am yet to test this. The only use I had for TypeScript was its typing system, in conjuction with JSDocs, to document types within the framework, a method which I documented about a while back.
You can read about integrating JSDocs and TypeScript for typing purposes, without the build process, here.
Code Splitting and Lazy Loading
Webpack is a powerful web development tool that allows for massively complex project configurations, giving development teams the flexibility that they need to build a project to its unique spec.
Oats~i runs on top of Webpack, with the framework primarily relying on Webpack's code splitting and lazy loading feature to support async fragment chunks and bundles.
This means your fragments can be loaded in one bundle or split using webpack into multiple chunks, optimizing initial load speeds for your Oats~i web app. Pair this with network-sourced views, if your app needs them, and there are multiple ways you can optimize your app in Oats~i to ensure the best user experience as far as loading times are concerned.
Advanced Project Configurations Using Webpack
Perhaps the biggest advantage of having webpack as a base for Oats~i is the large configuration left at your disposal, allowing you to craft your app as you need it.
That's why you can set up templating engines that suit your view rendering process, configure babel and other loaders/plugins for your app, and simply build something that is fully-specced to your project's specifics.
Oats~i runs a simple base webpack configuration that sets up handlebars-loader, html-loader, css loader, asset loader, and HTMLWebpackPlugin to create your server-side views or templates. Using webpack-merge, you can extend these configurations and architect your web app as you want it.
This makes Oats~i works a lot like a plug-and-play system. It gives you a skeleton, and you can wrap and configure your app around it as you like.
Routing
Routing is a default feature in Oats~i. In fact, to run the app, you must provide routing information that the app will use to initialize itself and manage user navigation and fragment rendering.
A simple routing information looks like this:
Const MyRoutingInfos = [
{
route: "/my-route",
target: myMainFragmentBuilder,
nestedChildFragments: [
myNestedChildFragmentBuilder
]
}
]
When Oats~i loads from the server, it checks the current url and finds a match for it in the provided routing info. In Verte's case, when you load "/", Oats~i searches for the routing info with that that route as a match and then inflates the fragments in order from "target" to each nested child fragment.
You can also provide a default route that Oats~i will try to start the app from, unless the client had sourced the page from a valid route given in your routing info.
Params in Routing
Oats~i also supports the use of params in routes, using the colon syntax commonly used in express.
Therefore, a route defined like /:myParams is valid, and will map for routes such as /user-1, /user-2, /user-3.
Oats~i goes a step farther and parses these params for you.
When setting up your fragment, you have the option of setting up params it should watch out for. The name of the param should be an EXACT match to the name used in your routing info.
When building the fragment, Oats~i will parse the value, note any changes, and pass two arguments to your onQueryParamsDataUpdate() method. These are an object of all watched params that have changed, and the current value of all watched params.
Therefore, if you have a fragment that shows user information, defined under the route /:userID, and the client first navigates to /user-xyz, you'll be able to read the value of userID as user-xyz. If the client routes again and this time the route is /user-abc, you'll immediately know that the value of userID has changed to user-abc and you can respond appropriately.
Queries Support
Queries are also a core part of web browsing and urls. Oats~i also parses queries for you, as long as you tell the fragment to watch them, using their keys.
For instance, if your route /:userID maps to /user-3?promptUpgrade=true, and you specify in your fragment that you want to watch updates for the query with the key "promptUpgrade", these will be parsed and sent to the method onQueryParamsDataUpdate() as well.
However:
You cannot write routes in your routing info using queries. Only params are supported. Oats~i looks for the valid routing info for a given url after truncating any queries and targets. The parsing will be done afterwards.
Verte's website already uses this mechanism when rendering views for blog articles at the blog article page. The route for each article is parameterized and we only respond to a change in the watched param.
Consent-Based Routing
This is perhaps a very unique feature from Oats~i. Consent-based routing gives you power over the user experience, allowing you to warn users about navigating away from a crucial page in case there are any pending processes, all controlled in-app.
Instead of using the provided standard browser API that pops up a dialog box, Oats~i uses a mix of History API and state management to detect a pop or navigation, ask the current rendered fragments for consent, halt subsequent navigation attempts, and proceed only if the user grants it permission.
If the user chooses to remain in their current view, Oats~i restores the browser's navigation path to the original state.
Of course, having users click on "ok" every time they want to navigate around your app is a bad idea. So, by default, Oats~i fragments and view panels (more on these later) consent to a navigation attempt by default.
Verte internally uses this to safeguard the admin when curating blog content, in case the current draft has not yet been picked up by the autosave script within its time delta. In case the admin wants to navigate away from the blog editor and there are unsaved drafts, they'll get a warning through a dialog and choose to either continue navigating away or stay on the page and manually save their work.
Pop-Ups, Dialogs, and More Layouts Using View Panels
In Oats~i, the framework will primarily render a route through fragments. However, there's an extra utility called view panels that allows you to render other views that your fragment may need on the fly. These include dialog boxes, hamburger panels, or even loading screens with bespoke information that the user may need.
To spawn a view panel, you have to request for it through the view panels manager. Oats~i self manages views for fragments and view panels, meaning you never have to write logic to bind your primary fragment views to the DOM or remove them once a view panel or its associated fragment is being destroyed due to a change in navigation.
A view panel, spawned by a view panels manager is also automatically wired into the consent-routing process of the fragment, allowing you to extend fragment functionality.
View panels can also watch params and queries.
Route-Triggered and Direct-Triggered View Panels
View panels can be triggered either by route changes or directly via a call to the fragment's view panels manager. For the former, this is where having queries in your route and linking them to a view panel within the fragment can come in handy.
If you have a route "/:post-id" which is currently represented in the browser as "/nice-post?showComments=true", you can use a route-triggered view panel within the fragment to automatically pop a side panel that loads the post comments and allows the user to read through them.
This feature is typically accessible through the onQueryParamsDataUpdate() method. Calling super (in case you've overridden it) will invoke the fragment's view panels manager to attempt to render any route-triggered view panels.
The biggest advantage of this kind of setup is that your view panel's rendering and behavior is now tied to the navigation, making the user experience more natural.
So, given our example, if the user navigated to "/nice-post?showComments=true", read the comments, and pressed back, the route will change back to "/nice-post", the view panels manager will note this change, and automatically trigger the destruction process for the view panel as long as consent has been granted.
Just like fragments, view panels also grant consent by default. Therefore, you should override the consent method ONLY when necessary.
Reactivity and Data Management
A modern web framework is not complete without a good touch of reactivity and data management. And here's where perhaps the most crucial difference between Oats~i and other web frameworks comes in.
Oats~i doesn't automatically couple views to a piece of data or state.
Instead, this is left entirely to the developer to do it based on their app or business logic.
As is, you can use Oats~i to build a web app with multiple static pages rendered under fragments and view panels and end it at that. The app will just work. If you want to add data, network calls, and reactivity, the data manager utility covers everything, and only to the scope that you determine, without affecting any surrounding views or data.
Let's look at the data manager and its supporting utilities: the network interface and view managers.
The Data Manager
The data manager is an Oats~i utility that allows you to tie data, server-resources, and client views together. The data manager holds an array of models, a model being the core piece or type of data associated with a section of your app and its selected views.
Currently, I've designed it to take a model as an object with arrays nested within, as it's the most common format for passing data around client and server resources (as Json).
Therefore, a simple model can look something like this:
{
my: string,
simple: number,
obj: {
ofArrays: number[],
objArrays: { objKey: string }[]
}
}
The data manager works by scoping its model. This means that every bit of the model can be treated as a unit, creating a set of dot-separated keys that define a specific value or type in your data.
For instance, in the example above, the data manager will break down the model into the following scopes: "MODEL_ROOT | my | simple | obj | obj.ofArrays | obj.objArrays | obj.objArrays.array.objKey "
These scopes represent:
MODEL_ROOT -> {
my: string,
simple: number,
obj: {
ofArrays: number[],
objArrays: { objKey: string }[]
}
}
my -> string,
simple -> number
obj -> {
ofArrays: number[],
objArrays: { objKey: string }[]
}
obj.ofArrays -> number[]
obj.objArrays -> { objKey: string }[]
obj.objArrays.array.objKey -> string
You can treat these scopes as dot-separated paths to a distinct piece of data.
With these scopes, the data manager then gives you, the developer, fine-grained control of your data, allowing to assign a network interface or view manager(s) to any of these data.
Let's shallowly dive into what these two are.
Network Interface
In most apps (native or web), the data shown to the user is sourced from an outside resource, a server. Therefore, the internal model often needs an API interface that sits between itself and the external resource.
In Oats~i's case, the network interface will perform the CRUD operation you need in relation to the data held by the data manager and ensure both ends are in sync.
The network interface is defined as an object with three methods:
getReqBody()
This method gets the body of the request and other data such as method, address, headers, etc.
onDataLoadPostProcess()
Because the type of response data and the type of your internal model may vary, the network interface allows you to post-process the response and provide the final data in the data manager's model type.
onDataLoadError()
This method allows you to format the error response in case the network call fails.
Network Interface Scoping
API designs are varied, meaning, the addresses or routes used to make CRUD operations for a given piece of data can be different.
For instance, a social media app can have a different API for loading all posts, and each post running unique APIs to repost, like, or report the post.
Assuming such an architecture, using scoping within the data manager allows you to specify unique network interfaces for each scope.
For instance, you can have a network interface for the MODEL_ROOT network call (which will load the posts), "repost" network call, and any other call that can be scoped out of the structure of the model the data manager holds.
This gives you a whole unique way of viewing your data, breaking it down from one large resource with a common end point, to a collection of multiple data units that can be handled independently through the data manager.
A key thing to note here is that you can only have one network interface per scope, creating a single "endpoint" for every scoped piece of data in your model.
View Manager
Through the network interface, the data manager can now keep data in sync between its model and the server. Now what about displaying it to the user and, more importantly, showing them when it's changing?
That's where the view manager comes in.
View managers respond to mutations or changes happening to data held by the data manager, through a network operation or a direct in-app change.
Oats~i currently supports two types of view managers - a standard view manager and a list view manager.
A standard view manager is ideal for simple views with components that are not duplicated over a list. On the other hand, a list view manager is best for "complex" views with view components duplicated over a list.
Regardless of the type, a view manager will tell you of the following changes within a model or its scoped types:
onMutate()
This method fires when a data type of the scope is changing
onCommit()
This method fires when a mutation of the data type of the scope has been completed, thus committed
onCancel()
This method fires when a mutation of the data type of the scope has been cancelled
onError()
This method fires when a mutation of the data type of the scope has encountered an error, allowing you to retry
There's also the builder set of methods, which allow you to pass in a view (as a HTML string) inflated with your data. These methods also inform you of when the view has been attached or about to be detached, depending on the mutation.
These three methods are:
inflateRoot()
Gets the templated view as a string for the data provided
onViewAttach()
Calls when the view has been attached to the DOM
onViewDetach()
Calls when the view is about to be detached from the DOM
You can see the results of these interactions in the blog pages of Verte's website.
Using the combination of builder methods, root hooks, and component hooks, the data-driven dynamic views of the blog and blog article fragments can show loading animations when we're sourcing data from the network, show error messages in case of a failure, and populate the views once the new data from the network interface has been committed.
A view manager will also have component hooks which allow for even finer grained reactivity, with the elements of each view node.
For instance, using the model:
{
my: string,
simple: number,
obj: {
ofArrays: number[],
objArrays: { objKey: string }[]
}
}
And a view manager of the scope "MODEL_ROOT" (therefore the whole model), we can assume that the main view component showing the data of the MODEL_ROOT scope, has components within it that my show the specific data held in "my", "simple", "obj", or generally the child scopes of MODEL_ROOT.
Therefore, you can set up a component or element of your view to react to changes in these "child" values.
All these hook methods get a viewNode parameter passed to them by the view manager, so you always have a reference of which view node these data changes are associated with and query its components as you need.
However, you should not bother with removing these core view elements once they're no longer needed. The view manager handles that for you.
No Virtual DOM
Oats~i doesn't operate through a virtual DOM. Instead, the fragments, view panels, and view managers directly use the DOM APIs to insert or remove DOM elements.
After inserting your view component into the DOM, the view manager will provide you with its direct reference in the builder, root, and component hooks. Therefore, you can just directly add listeners, change attributes, or simply manipulate the DOM element using the direct DOM apis.
Lifecycle Management
A core bit of a complex web app is lifecycle management. Oats~i has its own lifecycle management process for fragments and view panels, whose functions are extended to other utilities such as the data manager, view managers, and remote request util (the actual utility the data manager uses in conjunction with the network interface to make network calls).
Therefore, straight off the bat, using Oats~i and its core utilities will have lifecycle automatically managed for you.
For instance, if you're using the data manager within a fragment to make a CRUD operation, and the user navigates away from the fragment, the data manager and remote request util will be able to cancel all network operations, skip updating view managers, and unregister them, because your fragment or view panel no longer exists.
Listening to Lifecycle Events
As an Oats~i developer, you can make use of a fragment or view panel's lifecycle management to create robust lifecycle-aware libraries that will work well in an Oats~i environment.
You just have to grab the lifecycle object using the internal method,
getLifeCycleObject()
and attach listeners to it. These listeners typically include four methods for:
onFragmentRunning()
Called when the fragment has been created and is running
onFragmentCancelled()
Called when the fragment's build has been cancelled
onFragmentDestroyed()
Called when the fragment has been destroyed
onViewReady()
Called when the fragment's view has been attached to DOM
*Note: *"Fragment" here also applies to view panels.
The main calls you need to watch out for are onFragmentRunning(), onViewReady(), and onFragmentDestroyed(). If your library adds functionality that is not UI-related, you can enable the library after getting the onFragmentRunning() call.
If the library manipulates views (such as an animation library), you can enable its functionality after receiving the onViewReady() call.
Once you get the onFragmentDestroyed() call, pack up, and stop everything.
OOP-Based Core
We have talk about a lot about some core features of Oats~i but we haven't talked about paradigm. How will you write core Oats~i code?
Well, Oats~i is an OOP-based web framework. That means most utilities are provided as classes. A fragment is created from an AppMainFragment or AppChildFragment class. The data manager is a class. View managers are classes and so on.
I chose OOP because of its reusability, garbage collection, and a much cleaner way of managing functions and processes within Oats~i.
For instance, no pun intended, having the fragment as a class allows Oats~i to do something clever. It never reconstructs the fragment class if it determines that its being reused. Instead, the build process just goes directly to firing onQueryParamsDataUpdate(), and doesn't re-render the fragment's main view or update that section of the DOM, since it's unnecessary.
Another advantage of doing this is that your fragment can retain part of its state amidst related route calls.
For instance, in Verte's case, when you're in the fragment that renders a blog article, clicking on another article under the "Other stories" list doesn't reconstruct the fragment. Instead, the original view is untouched, and only the dynamic, data-driven views, ran by the data manager in conjunction with the view manager, update based on the new param value obtained from onQueryParamsDataUpdate().
Exploiting Functional Programming
Just because the Oats~i core uses OOP, doesn't mean you're fully restricted to creating libraries that follow the OOP paradigm. Simply making them lifecycle aware is enough.
This will allow them to capture and free resources from the fragment as Oats~i renders and destroys them.
When porting Verte's client to Oats~i, I've used this exact strategy to reuse some functional scripts I'd written for the original webpages.
Therefore, I expect very few bottlenecks and paradigm strictness for developers seeking to use their functional scripts in an Oats~i project, as long as they're lifecycle aware.
Server-Side Rendering (Views and Data)
Finally, a big part of modern web frameworks - server-side rendering.
Oats~i natively supports server-side rendering, with no need for running JavaScript on the server.
Using HTMLWebpackPlugin, you can extract the views you use for each fragment in your Oats~i app into their own .html/.hbs files that you can send to the client when they request for a page on a fresh load.
The only requirement is your view structure from the server is the same as the app would have rendered it.
But we're not done yet.
Data Manager Hydration
The views you'll render from your server most likely will represent some state of data. How does Oats~i handle this and proceed from the server-side rendered state?
You'll ideally be using the data manager to manage dynamic or data-driven views in your Oats~i app. Now, using it, you can can leverage server-side hydration that uses a script rendered in the head tag from the server to help the data manager understand the data state from the server, save it, and have attached view managers also update their states based on it, and continue running the app from there.
Here's how it works.
In your markup's head, at the server, you can add a script of the following format:
<script id="hydrator_id">
const DataManagerHydrationInfo = {
"info_key": {
info: model[]
extras: *
}
}
window.DataManagerHydrationInfo = DataManagerHydrationInfo;
</script>
This script provides important information for the data manager from the server, that gives it the full picture or an idea of the data state.
Each data manager will have an "info_key" that it will read its data state from. Once you set the data manager to hydrate from server side, it will find the script with the id you provide, get the exposed variable DataManagerHydrationInfo, and read the value of "info_key".
This value should be an array, ideally of the same type as the data manager's model. However, it can be different.
That's because the data manager runs a multi-step hydration process.
Validation
Reading from a script can have its own issues and vulnerabilities. You can run a validation check on the data stored in the hydration script before the data manager commits it.
Preprocessing
Depending on your business logic and web app design, the data format sourced from your server can be different from the model you run in your data manager. Oats~i's data manager runs an optional preprocessing step during hydration, that allows you to convert the data from the hydrator to your model's format.
Network Step
This step permits you to be cautious with the data you let free in your hydration script, open to web scrapers, robots, and search engines.
You can run an optional network step where you can get private or hidden data that your data manager's model needs, but should never be privy to web scrapers or robots scouring the web.
For instance, if you're hydrating a shopping cart, you can have the hydration script from the server contain only general information about the products, with public ids that when passed to your secure backend, will return more secret information that you'll use to check-out the user.
So, your hydration script can hold information as basic as what is already rendered in the html, have the data manager commit that immediately internally, then source everything else from the network cycle.
What Next for Oats~i?
If you've managed to read up to this point, kudos, you're a champ! That's the best I could do to try and squeeze roughly four years of work into a small "introductory" blog post.
Oats~i has been a massive learning project for me and am both anxious and excited to let the tech community know about it. There's a lot to unpack, teach, learn, and debug.
My plan at the moment is to open source Oats~i. I'm working on the specifics and hopefully the whole codebase will drop in the next few days and we can all dig in, build actual web apps through the framework, and take it up its paces.
For now, I'll appreciate your feedback, comments, and questions concerning Oats~i, if you have any.
Check out Verte Environmental Solution's website and see it in action.
I'm available on LinkedIn, so drop by and say hi.
See you soon, when we'll, hopefully, start building with Oats~i.
EDIT: The source code is now public. https://github.com/Oats-i/Oats-i
