Cong LiGBase database (GBase数据库) can be applied in OLTP scenarios, supports centralized deployment, shared storage high availability deployment, and multi-site high availability deployment.
Today, we will explore the basic principles and setup steps for the GBase 8s RHAC (Remote High Availability Cluster). RHAC is a technology designed to run within an asynchronous communication framework, minimizing the impact on the primary server while providing high data availability.
Overview of GBase 8s RHAC
The GBase 8s RHAC secondary server is similar to the HAC (High Availability Cluster) secondary server in many ways. The method for sending logs to the RHAC secondary server is similar to how the primary server sends logs to the HAC secondary server. However, the RHAC secondary server operates entirely within an asynchronous communication framework, minimizing the impact on the primary server. Transaction commitment and checkpoints are not synchronized between the primary server and the RHAC secondary server. Therefore, there is no guarantee that any transaction committed on the primary server will also be completed on the RHAC secondary server at the same time.
When comparing the functionality of RHAC and HAC, RHAC lacks the following features supported by HAC:
- Does not support full synchronization and near-synchronization modes
- Does not support the automatic switch of the DRAUTO parameter
- Does not support synchronized checkpoints
RHAC Principles
RHAC uses SMX (Server Multiplexer) communication to establish network connections between nodes. SMX supports full-duplex communication protocols and asynchronous communication, providing better connectivity over slow networks, simplifying communication management between nodes, supporting encrypted transmission, and allowing multiple internal functions over the same SMX connection.
The RHAC_Send thread sends logical logs to the RHAC secondary node. If the next page of logical logs is not in the logical log buffer, RHAC_Send will directly read the logs from the disk. The RHAC_Send thread interacts with SMX to send data in full-duplex mode. With full-duplex communication, the thread does not wait for acknowledgement from the secondary server before sending the next buffer. The primary server can send up to 32 buffer transmissions before waiting for acknowledgement from the RHAC secondary server. If the 32-buffer limit is reached, the sending thread will wait for the RHAC_Recv thread to receive an acknowledgement from the RHAC secondary server. On the secondary server, RHAC_Recv interacts with SMX to receive the log pages from the primary server.
The RHAC architecture simply relies on communication between the primary and secondary nodes to achieve data transmission and recovery. When data is generated, it is first written to the primary node's logical log buffer and storage. Simultaneously, the data is written to the sending buffer. To ensure data consistency, dedicated threads transmit the data from the sending buffer over the network to the secondary node. Once the secondary node receives the data, it stores it in its logical log buffer and uses replay techniques to update the values in memory.
A key feature of the RHAC architecture is the support for delayed data recovery. Delayed data recovery means that when the secondary node receives the data, it does not immediately perform the replay operation. Instead, it delays the operation for a certain period based on predefined rules. This delayed recovery mechanism can be implemented by caching data in temporary folders or other storage locations.
Data transmission in the RHAC architecture is asynchronous. The secondary node monitors the data differences between the primary and secondary nodes to control traffic flow. If the data discrepancy becomes too large, the secondary node will notify the primary node to take appropriate business actions.
RHAC Deployment
The deployment of RHAC clusters is very similar to that of HAC clusters.
| RHAC | HAC |
|---|---|
| Primary Node (192.168.1.1) | Secondary Node (192.168.1.2) |
| Disable firewall | Disable firewall |
| Disable SELinux | Disable SELinux |
| Adjust OS parameters (shared memory, semaphores, resource limits, etc.) | Adjust OS parameters (shared memory, semaphores, resource limits, etc.) |
| Configure node mutual trust | Configure node mutual trust |
| Install GBase 8s and initialize instance (primary) | Non-instantiated installation of GBase 8s |
| - | Configure environment variables |
| Configure onconfig parameters | Configure onconfig parameters |
| Adjust sqlhosts, add secondary node information | Adjust sqlhosts, same as primary node |
| Restart primary node instance | - |
| Register RHAC node on primary node (onmode -d add RSS ) | - |
| Perform L0 backup | - |
| - | Perform L0 backup recovery |
| - | Register secondary node to primary node (onmode -d RSS ) |
| Check node status (onstat -g rss) | Check node status (onstat -g rss) |
Install GBase 8s
First, install GBase 8s on both the primary and secondary nodes. The installation process is not elaborated here. Note: After installation, the primary node requires instance initialization, while the secondary node does not.
Modify the Primary Node's Configuration
After initializing the primary node's instance, modify the onconfig file. SERVERNUM and DBSERVERNUM can remain unchanged (as long as they are different between the primary and secondary nodes). The other parameters, after initialization, will have default values and must be modified. The DRAUTO parameter controls failover behavior. This parameter governs how the secondary node becomes the primary node in the event of a failure. Below is a table of the configuration parameters:
| Parameter | Value | Description |
|---|---|---|
| SERVERNUM | 131 | Unique instance identifier (max value is 255) |
| DBSERVERNAME | prim | Instance name |
| HA_ALIAS (Optional) | prim | Instance alias in high-availability environments |
| DRAUTO | 0 or 3 | 0: No automatic failover; 3: Failover controlled by CM |
| LOG_INDEX_BUILDS | 1 | Required: Writes index data to the logical log |
Next, modify the sqlhosts file to include the secondary node's information, as shown below:
[gbasedbt@prim ~]$ cd /opt/gbase/etc/
[gbasedbt@prim etc]$ cat sqlhosts.prim
prim onsoctcp 192.168.56.201 9088
[gbasedbt@prim etc]$ vi sqlhosts.prim
[gbasedbt@prim etc]$ cat sqlhosts.prim
prim onsoctcp 192.168.1.1 9088
rhac onsoctcp 192.168.1.2 9088
After modifying the configuration file, restart the primary node to apply the changes.
Modify the Secondary Node's Configuration
On the secondary node, modify the onconfig file to make SERVERNUM, DBSERVERNAME, and HA_ALIAS parameters differ from the primary node. Here are the values:
| Parameter | Value | Description |
|---|---|---|
| SERVERNUM | 132 | Unique instance identifier (max value is 255) |
| DBSERVERNAME | rhac | Instance name |
| HA_ALIAS (Optional) | rhac | Instance alias in high-availability environments |
| DRAUTO | 0 or 3 | 0: No automatic failover; 3: Failover controlled by CM |
| LOG_INDEX_BUILDS | 1 | Required: Writes index data to the logical log |
Modify the profile file to replace prim with rhac:
[gbasedbt@rhac ~]$ cat profile.rhac
export GBASEDBTSERVER=rhac
export GBASEDBTDIR=/opt/gbase
export GBASEDBTSQLHOSTS=/opt/gbase/etc/sqlhosts.rhac
export ONCONFIG=onconfig.rhac
export PATH=/opt/gbase/bin:$PATH
export PATH=/opt/gbase/sbin:$PATH
export DB_LOCALE=zh_CN.utf8
export CLIENT_LOCALE=zh_CN.utf8
export GL_USEGLU=1
After completing these changes, source the environment variables and check the database service status on the secondary node. At this point, the secondary node should be in a shutdown state.
[gbasedbt@rhac ~]$ source profile.rhac
[gbasedbt@rhac ~]$ onstat -
Your evaluation license will expire on 2024-11-20 00:00:00
shared memory not initialized for GBASEDBTSERVER 'rhac'
Creating a Chunk on the Secondary Node
First, check the storage path for the dbs on the primary node, and then create the same folder on the secondary node.
[gbasedbt@prim rhac_dbs]$ ll
total 8527872
-rw-rw---- 1 gbasedbt gbasedbt 104857600 Nov 21 10:00 datadbs1_1
-rw-rw---- 1 gbasedbt gbasedbt 3481272320 Nov 21 10:28 llogdbs
-rw-rw---- 1 gbasedbt gbasedbt 3481272320 Nov 21 10:28 plogdbs
-rw-rw---- 1 gbasedbt gbasedbt 209715200 Nov 21 10:28 rootdbs
-rw-rw---- 1 gbasedbt gbasedbt 727711744 Nov 21 09:58 sbspace1
-rw-rw---- 1 gbasedbt gbasedbt 727711744 Nov 21 09:58 tmpdbs1
[gbasedbt@prim rhac_dbs]$ pwd
/opt/gbase/rhac_dbs
Create the rhac_dbs directory on the secondary node under /opt/gbase, then create chunks on the secondary node with the same names as those on the primary node. As shown above, the primary node has six chunks, so we will create six chunks on the secondary node as well, and set their permissions to 660.
[gbasedbt@rhac ~]$ cd /opt/gbase/
[gbasedbt@rhac gbase]$ mkdir rhac_dbs
[gbasedbt@rhac gbase]$ cd rhac_dbs/
[gbasedbt@rhac rhac_dbs]$ touch datadbs1_1 llogdbs plogdbs rootdbs sbspace1 tmpdbs1
[gbasedbt@rhac rhac_dbs]$ ll
total 0
-rw-rw-r-- 1 gbasedbt gbasedbt 0 Nov 21 10:37 datadbs1_1
-rw-rw-r-- 1 gbasedbt gbasedbt 0 Nov 21 10:37 llogdbs
-rw-rw-r-- 1 gbasedbt gbasedbt 0 Nov 21 10:37 plogdbs
-rw-rw-r-- 1 gbasedbt gbasedbt 0 Nov 21 10:37 rootdbs
-rw-rw-r-- 1 gbasedbt gbasedbt 0 Nov 21 10:37 sbspace1
-rw-rw-r-- 1 gbasedbt gbasedbt 0 Nov 21 10:37 tmpdbs1
[gbasedbt@rhac rhac_dbs]$ chmod 660 *
[gbasedbt@rhac rhac_dbs]$ ll
total 0
-rw-rw---- 1 gbasedbt gbasedbt 0 Nov 21 10:37 datadbs1_1
-rw-rw---- 1 gbasedbt gbasedbt 0 Nov 21 10:37 llogdbs
-rw-rw---- 1 gbasedbt gbasedbt 0 Nov 21 10:37 plogdbs
-rw-rw---- 1 gbasedbt gbasedbt 0 Nov 21 10:37 rootdbs
-rw-rw---- 1 gbasedbt gbasedbt 0 Nov 21 10:37 sbspace1
-rw-rw---- 1 gbasedbt gbasedbt 0 Nov 21 10:37 tmpdbs1
At this point, the installation, initialization, and configuration file modifications are complete.
Configuring Mutual Trust
When deploying a GBase 8s cluster, there are various methods for configuring mutual trust. This article focuses on the method using the .rhosts file.
The .rhosts file in the user's home directory contains trusted {host ~ user} relationship pairs. It grants a specific user on a particular host mutual trust. Simply put, .rhosts provides trust between the same user across different hosts and only trusts the corresponding user on the trusted host.
This .rhosts method is suitable for environments where OS security requirements are low and trust needs to be specified at the user level. The root user is strictly managed in such cases. Here, we will use the gbasedbt user to modify the .rhosts file for configuring mutual trust.
Configuration Steps
First, modify the /etc/hosts file on both the primary and secondary nodes as the root user by running:
vi /etc/hosts
Add the IP addresses and hostnames for both nodes. The configuration should be the same on both servers.
[root@localhost opt]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.1 prim
192.168.1.2 rhac
After modifying the /etc/hosts file, create and modify the .rhosts file in the gbasedbt user’s home directory. Both the primary and secondary nodes need to add each other's hostname and mutual trust user (gbasedbt) in the file.
[gbasedbt@hac ~]$ cat .rhosts
prim gbasedbt
hac gbasedbt
Finally, configure the sqlhosts file correctly to complete the mutual trust configuration. The sqlhosts configuration has already been completed earlier, and at this point, mutual trust is fully configured.
Setting Up the RHAC Cluster
After completing the preparation work for the cluster, you can start setting up the cluster relationships.
First, configure the primary node, using rhac as the name of the secondary node instance. Then, you can check the cluster status, and you will see that rhac is in the Disconnected state.
[gbasedbt@prim rhac_dbs]$ onmode -d add RSS rhac
Your evaluation license will expire on 2024-11-20 00:00:00
[gbasedbt@prim rhac_dbs]$ onstat -g cluster
Your evaluation license will expire on 2024-11-20 00:00:00
On-Line -- Up 00:00:47 -- 674664 Kbytes
Primary Server:prim
Current Log Page:7,4081
Index page logging status: Enabled
Index page logging was enabled at: 2023/11/21 10:39:46
Server ACKed Log Applied Log Supports Status
(log, page) (log, page) Updates
rhac 0,0 0,0 No ASYNC(RSS),Disconnected,Defined
Next, perform data recovery on the secondary node, ensuring that the secondary node is in a shutdown state before executing the following command. On the primary node, run:
[gbasedbt@rhac ~]$ onstat - # Secondary node
Your evaluation license will expire on 2024-11-20 00:00:00
shared memory not initialized for GBASEDBTSERVER 'rhac'
On the primary node:
[gbasedbt@prim rhac_dbs]$ ontape -s -L 0 -t STDIO | ssh 192.168.56.202 "source /home/gbasedbt/profile.rhac && ontape -p -t STDIO"
Your evaluation license will expire on 2024-11-20 00:00:00
gbasedbt@192.168.56.202's password:
Your evaluation license will expire on 2024-11-20 00:00:00
After executing this command on the primary node, you can check the secondary node's status, and it will switch to the Fast Recovery state.
[gbasedbt@rhac ~]$ onstat - # Secondary node
Your evaluation license will expire on 2024-11-20 00:00:00
Fast Recovery -- Up 00:01:47 -- 597864 Kbytes
Now, we need to set the secondary node. Run the following command on the secondary node. After execution, you will notice two changes in the secondary node's status. First, it will switch to the Fast Recovery (RSS) state, then it will change to the Read-Only (RSS) state. You can check the cluster status, and rhac will be in the Connected state.
[gbasedbt@rhac ~]$ onmode -d RSS prim
Your evaluation license will expire on 2024-11-20 00:00:00
[gbasedbt@rhac ~]$ onstat -
Your evaluation license will expire on 2024-11-20 00:00:00
Fast Recovery (RSS) -- Up 00:02:27 -- 1135464 Kbytes
[gbasedbt@rhac ~]$ onstat -g cluster
Your evaluation license will expire on 2024-11-20 00:00:00Read-Only (RSS) -- Up 00:02:57 -- 1135464 Kbytes
Primary Server:prim
Index page logging status: Enabled
Index page logging was enabled at: 2023/11/21 10:39:46
Server ACKed Log Supports Status
(log, page) Updates
rhac 7,4092 No ASYNC(RSS),Connected,Active
The RHAC cluster relationship setup is now complete.
Through this article, we have learned the basic principles and setup steps of RHAC of GBase database (GBase 数据库). As an efficient database high-availability solution, RHAC not only provides asynchronous data replication but also maintains efficient communication even under suboptimal network conditions. With the continuous advancement of technology, RHAC will continue to provide strong support for data security and business continuity in enterprises. Thank you for reading.
