Yassine SallamiWorking with databases is inevitable in any modern software project. However, hardcoding database connections and operations in your main code can lead to poor scalability, difficult maintenance, and security issues. To combat these challenges, consider creating a Config Manager to handle all interactions with your database efficiently.
In this post, we'll walk through the advantages of using a Config Manager and how it can be implemented. Let’s start by looking at an example in Python using MongoDB and Streamlit.
Why You Shouldn't Call Your Database Directly
When your application grows, having direct database calls littered throughout the code can cause:
Hard to manage changes: If you need to change database settings or schema, you have to search through your codebase to modify each instance.
Lack of centralized error handling: Error-prone operations, such as connecting to the database or handling connection timeouts, become harder to manage when scattered.
Tight coupling: Direct database operations in the main code introduce tight coupling, making the code harder to test and refactor.
To avoid these issues, you should encapsulate database operations inside a Config Manager.
What is a Config Manager?
A Config Manager is a class or module that centralizes all database operations, handles error management, and makes your code more modular. It abstracts the details of the database, providing clean methods to interact with it.
Example: MongoDB Config Manager
import streamlit as st
from pymongo import MongoClient
from pymongo.errors import ServerSelectionTimeoutError, AutoReconnect
from dotenv import load_dotenv
import os, datetime
load_dotenv()
# Get MongoDB URI from environment variables
mongo_uri = os.getenv("MONGODB_URI")
class ConfigManager:
def __init__(self, user_id: str, db_name: str = "your_database", collection_name: str = "your_collection") -> None:
self.user_id = user_id
self.client = self.connect_to_mongo(mongo_uri)
if self.client:
self.db = self.client[db_name]
self.collection = self.db[collection_name]
self.user = self.collection.find_one({"badge_id": self.user_id})
self.create_date = datetime.datetime.now()
self.formatted_datetime = self.create_date.isoformat()
if not self.user:
self.add_new_user()
self.user = self.collection.find_one({"badge_id": self.user_id})
else:
st.toast(":red[Failed to connect to MongoDB. User initialization aborted.]")
def connect_to_mongo(self, uri, retries=3):
client = None
for attempt in range(retries):
try:
print(f"Attempting to connect to MongoDB (Attempt {attempt + 1})")
client = MongoClient(uri, serverSelectionTimeoutMS=20000)
client.server_info() # Will raise an exception if unable to connect
print("Connection to MongoDB successful.")
return client
except ServerSelectionTimeoutError as err:
print(f"Connection attempt {attempt + 1} failed: {err}")
except AutoReconnect as err:
print(f"AutoReconnect error on attempt {attempt + 1}: {err}")
print(f"Failed to connect to MongoDB after {retries} attempts.")
return None
def add_new_user(self):
new_user = {
"badge_id": self.user_id,
"created_at": self.formatted_datetime,
"badge_usage": 0,
"created_docs": 0,
"limitation": 10,
"doc_params": self._get_default_doc_params(),
"security": self._get_default_security(),
"user_activity": {"vault": [], "action_merge": []}
}
self.collection.insert_one(new_user)
def _get_default_doc_params(self) -> dict:
return {
"includes": ["upper", "lower", "number", "symbol", "arabic"],
"input": {"length": {"max": 13, "min": 3}, "types": ["text", "image", "pdf", "audio", "video"]}
}
def _get_default_security(self) -> dict:
return {"encryption": "cryptography.fernet", "encrypted_data": "json format", "file_extension": ".dkp"}
Key Features of the Config Manager
Centralized Database Handling: The ConfigManager is responsible for connecting to the MongoDB database. By abstracting this logic, you ensure that the rest of the application doesn’t need to worry about connection details or handling retries.
Automatic User Initialization: When a user (in this case identified by user_id) interacts with the application, the manager checks if the user exists in the database. If not, it automatically creates a new user record, maintaining consistency in your database operations.
Retry Logic: The
connect_to_mongo()method includes a retry mechanism for connection failures. This reduces the likelihood of total failure and ensures resilience in unstable network environments.Separation of Concerns: The rest of the application doesn’t need to know about database internals, reducing dependencies and making it easier to modify database behavior in the future.
Benefits of Using a Config Manager
Easier Maintenance: By having all database interactions in one place, it’s easier to update or debug any issues related to the database.
Error Handling: With a Config Manager, you can define custom error-handling logic that applies to all database interactions, avoiding unexpected crashes due to database timeouts or connection issues.
Scalability: When your project grows, you won’t need to refactor code scattered across different modules. You can easily add new database operations or modify existing ones in the manager.
Enhanced Security: Database URIs and other sensitive information are abstracted away from the main code and retrieved securely using environment variables.
Conclusion
A Config Manager is essential for building scalable and maintainable applications that interact with databases. It allows you to centralize connection logic, error handling, and security concerns. By abstracting these details away from the main code, you create a cleaner, more modular, and future-proof project structure.
Instead of hardcoding database operations in your main application, invest in building a Config Manager and see the difference it makes in your code’s cleanliness and maintainability.
Thank you for reading my post :)
