Create PDBs in a Data Guard Environment with TDE enabled

In this blog post we will see what happens and how to proceed after creating or cloning a PDB in a Data Guard environment while Transparent Data Encryption (TDE) is enabled.

If you are new to TDE, have a look at this blog first.

The Environment

I'm using Oracle Database Cloud service Enterprise Edition Extreme Performance. The database version is 19.8. TDE is enabled by default. After creating a DBCS system on a virtual machine, I just enable Data Guard from the UI to create the standby database. I ended up with the following configuration (screenshots in purple are from DGMGRL, turquoise from the primary, and gray from the standby):

dgmgrl
connect sys;
show configuration;

The auto-login wallet is open and PDB1 has a master encryption key:

set lines 300
col name for a15
col wrl_type for a10
col status for a20
select p.con_id, p.name, p.open_mode, ew.wrl_type, ew.wallet_type, ew.status 
from v$pdbs p join v$encryption_wallet ew on (ew.con_id = p.con_id)
order by p.con_id;


set lines 300
col name for a10
col key_id for a60
col creation_time for a40
select p.con_id, p.name, p.open_mode, ek.key_id --, ek.creation_time, ek.KEY_USE
from v$pdbs p left outer join v$encryption_keys ek on (ek.con_id = p.con_id)
order by p.con_id;

The wallet files on primary and standby are the same (have the same timestamp). Check the wallet location in the sqlnet.ora or the wallet_root database parameter:

!ls -ltr /opt/oracle/dcs/commonstore/wallets/tde/CDBPRIM_fra1g6/*wallet.*
!ls -ltr /opt/oracle/dcs/commonstore/wallets/tde/CDBPRIM_fra233/*wallet.*

The Managed Recovery Process (MRP) is running on standby:

select status, blocks, delay_mins, known_agents 
from gv$managed_standby where process='MRP0';

So everything is fine so far.

Step 1: Create a new PDB on primary

Let's create a new pluggable database on the primary:

create pluggable database PDB2 admin user newpdbadmin identified by NewAdPW_12;
alter pluggable database PDB2 open;

Check the wallet status on primary and standby:

select p.con_id, p.name, p.open_mode, ew.wrl_type, ew.wallet_type, ew.status 
from v$pdbs p join v$encryption_wallet ew on (ew.con_id = p.con_id)
where p.name = 'PDB2';

The wallet is open, but there is no master encryption key in the new PDB yet. So we are going to create it. MRP is running and everything is fine (believe me without a screenshot).

Step 2: Create a master encryption key in the new PDB

Now we switch to the PDB and create the master encryption key there. The wallet password is the same password you have chosen for the SYS user while DBCS system creation:

alter session set container=pdb2;
administer key management set key force keystore identified by MySysPW_12 with backup;

Check STATUS again:

The master encryption key is created in the primary, but not in the standby.

Check the wallet files on the file system:

As we see, the wallet files on the primary system got a newer timestamp, but not on standby.

Also, the MRP process stopped, and the data guard configuration is showing an error:

This happens after the "administer key management" SQL command on the primary.

Step 3: Copy the wallet files to the stand by

To solve the situation, first, we copy the wallet files to the standby:

!scp -p /opt/oracle/dcs/commonstore/wallets/tde/CDBPRIM_fra1g6/cwallet.sso oracle@130.61.227.63:/opt/oracle/dcs/commonstore/wallets/tde/CDBPRIM_fra233/
!scp -p /opt/oracle/dcs/commonstore/wallets/tde/CDBPRIM_fra1g6/ewallet.p12 
oracle@130.61.227.63:/opt/oracle/dcs/commonstore/wallets/tde/CDBPRIM_fra233/

The wallet files on the standby system are updated now. But the master encryption key on the standby is still not recognized:

Step 4: Close and reopen the wallet on the stand by

For the master encryption key in the new PDB to get recognized on standby, we have to close and reopen the wallet on standby:

administer key management set keystore close container=ALL;

select p.con_id, p.name, p.open_mode, ew.wrl_type, ew.wallet_type, ew.status 
from v$pdbs p join v$encryption_wallet ew on (ew.con_id = p.con_id)
where p.name = 'PDB2';

The auto-login wallet opens automatically as soon as we query the v$encryption_wallet view. The wallet status on standby has been changed to OPEN now.

The MRP process is still not running:

Step 4: Start the MRP process

The last step is to start the MRP process:

DGMGRL> edit database CDBPRIM_fra233 set state=APPLY-ON;

And one very last check :-)

Check Oracle documentation before trying anything in a production environment.

Thanks for reading!