A two-node 19c RAC with Data Guard on an Apple silicon Mac

I now have a working two-node Oracle 19c RAC with a Data Guard physical standby running entirely on my MacBook Pro, an M5 with 48 GB of memory. It went from an empty Parallels install to broker status SUCCESS in one evening, and it survives a full stop and start without anyone touching it. This post covers how it is put together, how it was built, and the problems that stopped the build along the way, because those are the part worth reading.

Why build it at all

Data Guard is easy to practice on one box. Two databases on the same VM and you are in business. RAC is not. RAC needs at least two hosts, shared disks that both hosts can see at the same time, a private interconnect, VIPs, and a SCAN that resolves through DNS. Cloud VMs can do all of that, but I wanted something I could snapshot, break, roll back and record without a meter running.

I set five requirements. It had to be two real RAC nodes with Grid Infrastructure, ASM and a container database, not a single-instance shortcut. It needed a physical standby managed by the broker. The whole build had to be scripted and repeatable, so a rebuild is a command and not a weekend. Every milestone needed a snapshot so I can roll back to any point for testing or demos. And it had to be license clean, because a lab that quietly uses Active Data Guard or the Diagnostics Pack teaches bad habits that follow people into production.

The lab at a glance

Four VMs run Oracle Linux 8.10 aarch64 and use about 34 GB of memory with everything up.

Three networks keep the traffic apart. The public network is the Parallels Shared NAT network on 10.211.55.0/24, which also gives the VMs internet access for dnf, and the SCAN name rac-scan.rac.lab resolves to .55, .56 and .57 on it. The interconnect runs on a host-only network, racpriv, on 192.168.100.0/24 with DHCP off, and it also carries ASM traffic. iSCSI gets its own host-only network, racstor, on 192.168.200.0/24, also with DHCP off.

The database is RACDB, a CDB with a single PDB named PDB1, in ARCHIVELOG mode. ASM has three disk groups. CRS uses NORMAL redundancy across three 5 GB disks and holds the OCR and three voting files. DATA uses two 25 GB disks and FRA uses one, both with EXTERNAL redundancy.

Design decisions

19c on ARM

Apple silicon runs ARM guests natively, and x86 emulation is far too slow for Clusterware. Oracle ships 19c for Linux on aarch64, so I used the 19.19 Grid Infrastructure and Database images on Oracle Linux 8.10 aarch64. If you plan to rely on RAC on ARM for anything beyond learning, check the certification matrix on My Oracle Support first.

A third VM for shared storage

Parallels has no true shared disk that two running VMs can attach at once, so a small storage VM exports file-backed LUNs over iSCSI using LIO and targetcli. The portal listens only on the storage network, and ACLs admit only the two RAC initiators. The LUN files are sparse, and write_back is off so that a write the nodes see as complete is actually on disk.

Stable ASM device names without ASMLib or AFD

Each node logs in to the target, and a udev rule keyed on the iSCSI ID_PATH, which encodes the portal, the target IQN and the LUN number, creates /dev/oracleasm/crs1 through fra1 owned by grid:asmadmin. The ASM discovery string is /dev/oracleasm/*. The names survive reboots and do not depend on which sdX device the kernel hands out on a given boot.

KERNEL=="sd*", SUBSYSTEM=="block", ENV{DEVTYPE}=="disk", \
  ENV{ID_PATH}=="ip-192.168.200.50:3260-iscsi-iqn.2026-09.lab.rac:stor.asm-lun-0", \
  SYMLINK+="oracleasm/crs1", OWNER="grid", GROUP="asmadmin", MODE="0660"

A real DNS server for the SCAN

The SCAN has to resolve to three addresses through DNS, not through /etc/hosts, or CVU complains and round robin does not work. dnsmasq on the storage VM serves every lab name from its own hosts file and forwards everything else upstream. The RAC nodes and the standby use it as their only resolver.

Deterministic NIC names

Every virtual NIC gets a fixed MAC address. A systemd .link file per MAC renames the interfaces pub0, priv0 and stor0, and a NetworkManager keyfile per name sets the static address. The Grid networkInterfaceList then reads the same on both nodes: pub0 public, priv0 ASM and private, stor0 do not use.

Role separation and a filesystem standby

A grid user owns the Grid home and ASM, and an oracle user owns the database home, which is how most production sites run. The standby is a single VM with no Grid Infrastructure and OMF datafiles on a local filesystem. That keeps the standby VM small, and it gives the lab a pattern I see in the field often: a RAC primary on ASM shipping redo to a single-instance standby on a filesystem.

The build, step by step

The whole build is driven from one configuration file and nine scripts on the Mac. Every step ends with a scripted verification and a Parallels snapshot, and every script is safe to rerun because finished sub-steps leave a marker and are skipped the next time.

From the first command to the last verification took about an hour and forty minutes, fixes included.

Unattended OS installs

The trick that makes the OS install hands-off is that Anaconda automatically loads ks.cfg from any volume labelled OEMDRV. Each VM gets two virtual CD drives: the Oracle Linux DVD and a tiny ISO holding that VM’s own kickstart. The kickstart sets the hostname, installs my SSH key for root, writes the .link and NetworkManager files, and powers the VM off when it finishes.

hdiutil makehybrid -o ks-rac1.iso ks/rac1 -iso -joliet \
  -default-volume-name OEMDRV -iso-volume-name OEMDRV -joliet-volume-name OEMDRV

prlctl create rac1 --ostype linux --no-hdd
prlctl set rac1 --cpus 4 --memsize 12288
prlctl set rac1 --device-add hdd --type expand --size 81920
prlctl set rac1 --device-add net --type shared --mac 001C421A1101
prlctl set rac1 --device-add net --type host-only --iface racpriv --mac 001C421A1102

When a VM powers off, the script detaches both ISOs, sets the boot order to the disk and starts it again.

Grid Infrastructure

The response file is the heart of the Grid install, and these are the lines that matter.

oracle.install.crs.config.gpnp.scanName=rac-scan.rac.lab
oracle.install.crs.config.clusterNodes=rac1.rac.lab:rac1-vip.rac.lab,rac2.rac.lab:rac2-vip.rac.lab
oracle.install.crs.config.networkInterfaceList=pub0:10.211.55.0:1,priv0:192.168.100.0:5,stor0:192.168.200.0:3
oracle.install.crs.config.storageOption=FLEX_ASM_STORAGE
oracle.install.crs.configureGIMR=false
oracle.install.asm.diskGroup.name=CRS
oracle.install.asm.diskGroup.redundancy=NORMAL
oracle.install.asm.diskGroup.disksWithFailureGroupNames=/dev/oracleasm/crs1,,/dev/oracleasm/crs2,,/dev/oracleasm/crs3,
oracle.install.asm.diskGroup.diskDiscoveryString=/dev/oracleasm/*

I skip the GIMR because a lab does not need it and it costs memory. gridSetup.sh runs silent as grid, then orainstRoot.sh and root.sh run strictly in order, rac1 first and then rac2, never in parallel. root.sh on rac1 took four and a half minutes. gridSetup.sh -executeConfigTools finishes the job.

The RAC database

DATA and FRA are created with plain SQL as SYSASM and started on both nodes with srvctl. The database home installs software-only from rac1 and the installer copies it to rac2. Then DBCA builds the database.

dbca -silent -createDatabase -templateName General_Purpose.dbc \
  -gdbName RACDB -sid RACDB -databaseConfigType RAC -nodelist rac1,rac2 \
  -createAsContainerDatabase true -numberOfPDBs 1 -pdbName PDB1 \
  -storageType ASM -datafileDestination +DATA \
  -recoveryAreaDestination +FRA -recoveryAreaSize 20000 -enableArchive true \
  -totalMemory 4096 -emConfiguration NONE \
  -initParams control_management_pack_access=NONE

Data Guard

On the primary I enabled force logging, set standby_file_management to AUTO, pointed the broker configuration files at +DATA and +FRA, and added standby redo logs, one more group than the online logs per thread and the same size. The password file comes out of ASM with asmcmd pwcopy and goes into the standby home’s dbs directory.

On the standby host I added a static listener entry for RACDBSB, wrote a one-line pfile with db_name set to RACDB, and started the instance NOMOUNT. RMAN then duplicates straight from the running primary.

duplicate target database for standby from active database
  spfile
    set db_unique_name 'RACDBSB'
    set cluster_database 'FALSE'
    set control_files '/u01/app/oracle/oradata/RACDBSB/control01.ctl','/u01/app/oracle/fra/RACDBSB/control02.ctl'
    set db_create_file_dest '/u01/app/oracle/oradata'
    set db_recovery_file_dest '/u01/app/oracle/fra'
    set db_recovery_file_dest_size '20G'
    set undo_tablespace 'UNDOTBS1'
    set fal_server 'RACDB'
  nofilenamecheck;

cluster_database has to go to FALSE, and the ASM-based control_files, audit_file_dest and listener settings have to be overridden, or the standby tries to come up as a RAC instance looking for +DATA. The broker comes last.

DGMGRL> create configuration 'raclab_dg' as primary database is 'RACDB' connect identifier is RACDB;
DGMGRL> add database 'RACDBSB' as connect identifier is RACDBSB maintained as physical;
DGMGRL> enable configuration;

What broke, and how I fixed it

No build of this size goes clean the first time. These are the problems I actually hit, and the first one is the one worth remembering.

The clock that starts in 2018

After the Grid snapshot I restarted the lab and Clusterware would not come up. GIPCD logged CRS-7507, failed to create its LISTEN endpoint, and CSS never started.

The root cause is that Parallels ARM guests have no real-time clock. /dev/rtc does not exist and hwclock fails. Without Parallels Tools in the guest, every boot starts at 2018-06-22, the systemd build date. chrony then jumps the clock forward eight years while OHAS is in the middle of starting, and the cluster stack does not recover from that. The same problem had already shown up earlier in a different disguise, when dnf failed with “Cannot download repomd.xml” because no TLS certificate is valid in 2018.

The fix for both is to get time right before anything depends on it. chrony-wait.service is enabled, chrony.conf uses makestep 1 -1 so the clock can be stepped at any time rather than only on the first updates, and OHAS is made to wait for time sync.

# /etc/systemd/system/oracle-ohasd.service.d/10-wait-for-time.conf
[Unit]
Wants=time-sync.target
After=time-sync.target chrony-wait.service

After that, a full stop and start brings the cluster up by itself in about a minute. If you run RAC on any hypervisor without a guest RTC, check this before you start blaming the interconnect.

libgfortran missing on aarch64

cluvfy reported one real failure, PRVG-2043, libgfortran-8.5.0 missing. The aarch64 preinstall RPM does not pull it in. It is the same kind of gap I hit with the 23ai Free install on OCI ARM, and the fix is the same: install it yourself before the installer finds it missing.

DBCA in 19.19 rejects -sidPrefix

DBCA failed with INS-04003, -sidPrefix is not a supported argument. In 19c the flag is -sid, and for a RAC database it becomes the instance prefix, which gives RACDB1 and RACDB2.

PDB1 came up MOUNTED after a restart

DBCA does not save PDB state. After the first restart, PDB1 sat in MOUNTED on both instances. Two statements, run once, fix it.

alter pluggable database PDB1 open instances=all;
alter pluggable database PDB1 save state instances=all;

Parallels CLI syntax

Parallels Desktop 27 does not accept —network for attaching a NIC to a custom network. It wants —type host-only —iface racpriv. It is a small thing, but it stops the build cold.

dnsmasq losing the SCAN after a reboot

With bind-interfaces and a listen-address, dnsmasq fails at boot if pub0 is not up yet, and then nothing resolves the SCAN. bind-dynamic binds the address as soon as it appears. I caught this one before it bit, because every snapshot restarts the whole lab.

The broker reports ERROR right after startup

After a restart, show configuration reports ORA-16810 and ORA-16854 for about 90 seconds, then SUCCESS once the standby catches up. That is expected behaviour, so the start script now waits for SUCCESS instead of raising an alarm on the first look.

Slow VM shutdowns

prlctl stop took two minutes per VM, because without Parallels Tools it waits for a timeout and then hard-stops the guest. The lab now shuts down from the inside: the standby first, then crsctl stop crs on each node, then a systemctl poweroff over SSH. A full lab stop takes about 26 seconds.

The finished lab

Every step ends with a scripted check rather than an eyeball. This is the Grid check on rac1.

PASS  CRS, CSS, EVM online on all nodes
PASS  two cluster nodes
PASS  three voting files
PASS  ASM running on both nodes
PASS  SCAN listeners running

The database check.

PASS  both instances running
PASS  archivelog mode
PASS  pack access is NONE
PASS  PDB1 open read write

And the broker, a few seconds after the standby finished its first apply.

Configuration - raclab_dg

  Protection Mode: MaxPerformance
  Members:
  RACDB   - Primary database
    RACDBSB - Physical standby database

Fast-Start Failover:  Disabled

Configuration Status:
SUCCESS   (status updated 14 seconds ago)

Transport lag and apply lag are both zero seconds, MRP is running, and the standby is a PHYSICAL STANDBY in MOUNTED mode. Then came the real test: a full lab stop, a full lab start, and every check passing again with no hands on the keyboard.

Running it day to day

Three commands cover normal use. Start brings up the storage VM first for DNS and iSCSI, then both RAC nodes, then the standby in MOUNTED mode. Stop runs in reverse, standby first and storage last. Status shows the VMs, Clusterware, the database, the standby role and the broker on one screen.

Every milestone has a Parallels snapshot taken with the lab cleanly stopped, from the fresh OS install through the finished Data Guard configuration. The last one is my clean starting point. I can kill a node, corrupt a datafile or run a failover, then roll all four VMs back in about a minute.

License hygiene

A lab is where habits form, so this one is built to stay inside Enterprise Edition without extra options. The standby stays MOUNTED. Opening a physical standby read only while redo apply runs is Real-Time Query, which is Active Data Guard, a separately licensed option, and my verification script fails if the standby is ever open read only. control_management_pack_access is NONE on both databases, so AWR, ASH and the tuning advisors stay off and the Diagnostics and Tuning Pack usage history stays clean. The duplicate uses no RMAN compression beyond BASIC, and nothing in the lab touches Advanced Security or Advanced Compression. I have written before about how the Diagnostics Pack becomes a license trap, and a lab is the easiest place to avoid building that habit.

What comes next

This lab is the base for the next few posts: switchover and failover with the broker, rolling patches across a RAC one node at a time, and what happens to a RAC primary when the interconnect drops. I will also use it to test how AI agents behave against a real RAC and Data Guard configuration, which is where a lot of Oracle shops are heading.

If your team runs RAC or Data Guard in production and wants a second set of eyes on availability, licensing exposure or agent access to your databases, get in touch. That is the work Sitratec does every day.