Ivan PesentiWelcome back, folks ๐! Today, we discuss a specific use case we might face when moving data back and forth from/to the database. First, let me set the boundaries for today's challenge. To stick to a real-life example, let's borrow some concepts from the U.S. Army ๐ช. Our deal is to write a small software to save and read the officers with the grades they have achieved in their careers.
Gorm's Custom Data Types
Our software needs to handle the army officers with their respective grades. At first look, it might seem easy, and we probably don't need any Custom Data Type here. However, to show off this feature, let's use a non-conventional way to represent the data. Thanks to this, we're asked to define a custom mapping between Go structs and DB relations. Furthermore, we must define a specific logic to parse the data. Let's expand on this by looking at the program's targets ๐.
Use Case to Handle
To ease things down, let's use a drawing to depict the relationships between the code and the SQL objects:
Let's focus on each container one by one.
The Go Structs ๐งฑ
Here, we defined two structs. The Grade struct holds a non-exhaustive list of military grades ๐๏ธ. This struct won't be a table in the database. Conversely, the Officer struct contains the ID, the name, and a pointer to the Grade struct, indicating which grades have been achieved by the officer so far.
Whenever we write an officer to the DB, the column grades_achieved must contain an array of strings filled in with the grades achieved (the ones with true in the Grade struct).
The DB Relations ๐
Regarding the SQL objects, we have only the officers table. The id and name columns are self-explanatory. Then, we have the grades_achieved column that holds the officer's grades in a collection of strings.
Whenever we decode an officer from the database, we parse the grades_achieved column and create a matching "instance" of the Grade struct.
You might have noticed that the behavior is not the standard one. We must make some arrangements to fulfill it in the desired way.
Here, the models' layout is overcomplicated on purpose. Please stick to more straightforward solutions whenever possible.
Custom Data Types
Gorm provides us with Custom Data Types. They give us great flexibility in defining the retrieval and saving to/from the database. We must implement two interfaces: Scanner and Valuer ๐ค. The former specifies a custom behavior to apply when fetching data from the DB. The latter indicates how to write values in the database. Both help us in achieving the non-conventional mapping logic we need.
The functions' signatures we must implement are Scan(value interface{}) error and Value() (driver.Value, error). Now, let's look at the code.
The Code
The code for this example lives in two files: the domain/models.go and the main.go. Let's start with the first one, dealing with the models (translated as structs in Go).
The domain/models.go file
First, let me present the code for this file:
package models
import (
"database/sql/driver"
"slices"
"strings"
)
type Grade struct {
Lieutenant bool
Captain bool
Colonel bool
General bool
}
type Officer struct {
ID uint64 `gorm:"primaryKey"`
Name string
GradesAchieved *Grade `gorm:"type:varchar[]"`
}
func (g *Grade) Scan(value interface{}) error {
// we should have utilized the "comma, ok" idiom
valueRaw := value.(string)
valueRaw = strings.Replace(strings.Replace(valueRaw, "{", "", -1), "}", "", -1)
grades := strings.Split(valueRaw, ",")
if slices.Contains(grades, "lieutenant") {
g.Lieutenant = true
}
if slices.Contains(grades, "captain") {
g.Captain = true
}
if slices.Contains(grades, "colonel") {
g.Colonel = true
}
if slices.Contains(grades, "general") {
g.General = true
}
return nil
}
func (g Grade) Value() (driver.Value, error) {
grades := make([]string, 0, 4)
if g.Lieutenant {
grades = append(grades, "lieutenant")
}
if g.Captain {
grades = append(grades, "captain")
}
if g.Colonel {
grades = append(grades, "colonel")
}
if g.General {
grades = append(grades, "general")
}
return grades, nil
}
Now, let's highlight the relevant parts of it ๐:
- The
Gradestruct only lists the grades we forecasted in our software - The
Officerstruct defines the characteristics of the entity. This entity is a relation in the DB. We applied two Gorm notations:-
gorm:"primaryKey"on theIDfield to define it as the primary key of our relation -
gorm:"type:varchar[]"to map the fieldGradesAchievedas an array ofvarcharin the DB. Otherwise, it translates as a separate DB table or additional columns in theofficerstable
-
- The
Gradestruct implements theScanfunction. Here, we get the raw value, we adjust it, we set some fields on thegvariable, and we return - The
Gradestruct also implements theValuefunction as a value receiver type (we don't need to change the receiver this time, we don't use the*reference). We return the value to write in the columngrades_achievedof theofficerstable
Thanks to these two methods, we can control how to send and retrieve the type Grade during DB interactions. Now, let's look at the main.go file.
The main.go file ๐ฆ
Here, we prepare the DB connection, migrate the objects to relations (ORM stands for Object Relation Mapping), and insert and fetch records to test the logic. Below is the code:
package main
import (
"encoding/json"
"fmt"
"os"
"gormcustomdatatype/models"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
func seedDB(db *gorm.DB, file string) error {
data, err := os.ReadFile(file)
if err != nil {
return err
}
if err := db.Exec(string(data)).Error; err != nil {
return err
}
return nil
}
// docker run -d -p 54322:5432 -e POSTGRES_PASSWORD=postgres postgres
func main() {
dsn := "host=localhost port=54322 user=postgres password=postgres dbname=postgres sslmode=disable"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
fmt.Fprintf(os.Stderr, "could not connect to DB: %v", err)
return
}
db.AutoMigrate(&models.Officer{})
defer func() {
db.Migrator().DropTable(&models.Officer{})
}()
if err := seedDB(db, "data.sql"); err != nil {
fmt.Fprintf(os.Stderr, "failed to seed DB: %v", err)
return
}
// print all the officers
var officers []models.Officer
if err := db.Find(&officers).Error; err != nil {
fmt.Fprintf(os.Stderr, "could not get the officers from the DB: %v", err)
return
}
data, _ := json.MarshalIndent(officers, "", "\t")
fmt.Fprintln(os.Stdout, string(data))
// add a new officer
db.Create(&models.Officer{
Name: "Monkey D. Garp",
GradesAchieved: &models.Grade{
Lieutenant: true,
Captain: true,
Colonel: true,
General: true,
},
})
var garpTheHero models.Officer
if err := db.First(&garpTheHero, 4).Error; err != nil {
fmt.Fprintf(os.Stderr, "failed to get officer from the DB: %v", err)
return
}
data, _ = json.MarshalIndent(&garpTheHero, "", "\t")
fmt.Fprintln(os.Stdout, string(data))
}
Now, let's see the relevant sections of this file. First, we define the seedDB function to add dummy data in the DB. The data lives in the data.sql file with the following content:
INSERT INTO public.officers
(id, "name", grades_achieved)
VALUES(nextval('officers_id_seq'::regclass), 'john doe', '{captain,lieutenant}'),
(nextval('officers_id_seq'::regclass), 'gerard butler', '{general}'),
(nextval('officers_id_seq'::regclass), 'chuck norris', '{lieutenant,captain,colonel}');
The main() function starts by setting up a DB connection. For this demo, we used PostgreSQL. Then, we ensure the officers table exists in the database and is up-to-date with the newest version of the models.Officer struct. Since this program is a sample, we did two additional things:
- Removal of the table at the end of the
main()function (when the program terminates, we would like to remove the table as well) - Seeding of some dummy data
Lastly, to ensure that everything works as expected, we do a couple of things:
- Fetching all the records in the DB
- Adding (and fetching back) a new officer
That's it for this file. Now, let's test our work ๐ธ.
The Truth Moment
Before running the code, please ensure that a PostgreSQL instance is running on your machine. With Docker ๐, you can run this command:
docker run -d -p 54322:5432 -e POSTGRES_PASSWORD=postgres postgres
Now, we can safely run our application by issuing the command: go run . ๐
The output is:
[
{
"ID": 1,
"Name": "john doe",
"GradesAchieved": {
"Lieutenant": true,
"Captain": true,
"Colonel": false,
"General": false
}
},
{
"ID": 2,
"Name": "gerard butler",
"GradesAchieved": {
"Lieutenant": false,
"Captain": false,
"Colonel": false,
"General": true
}
},
{
"ID": 3,
"Name": "chuck norris",
"GradesAchieved": {
"Lieutenant": true,
"Captain": true,
"Colonel": true,
"General": false
}
}
]
{
"ID": 4,
"Name": "Monkey D. Garp",
"GradesAchieved": {
"Lieutenant": true,
"Captain": true,
"Colonel": true,
"General": true
}
}
Voilร ! Everything works as expected. We can re-run the code several times and always have the same output.
That's a Wrap
I hope you enjoyed this blog post regarding Gorm and the Custom Data Types. I always recommend you stick to the most straightforward approach. Opt for this only if you eventually need it. This approach adds flexibility in exchange for making the code more complex and less robust (a tiny change in the structs' definitions might lead to errors and extra work needed).
Keep this in mind. If you stick to conventions, you can be less verbose throughout your codebase.
That's a great quote to end this blog post.
If you realize that Custom Data Types are needed, this blog post should be a good starting point to present you with a working solution.
Please let me know your feelings and thoughts. Any feedback is always appreciated! If you're interested in a specific topic, reach out, and I'll shortlist it. Until next time, stay safe, and see you soon!
