Tuesday, December 13, 2022

ORA-32010: cannot find entry to delete in SPFILE

1. SQL> alter system reset "_time_based_rcv_ckpt_target" scope = spfile sid='aaa'

 *
ERROR at line 1:
ORA-32010: cannot find entry to delete in SPFILE

2.
SQL> alter system reset "_bct_public_dba_buffer_size" sid='aaa' scope=spfile ;

System altered.

3. 😶

Thursday, December 1, 2022

DBMS_WORKLOAD_CAPTURE: cannot import STS for capture with ID... in the alert.log

This error encountered in the alert log file of  replay system. The replay was being using to assess the consequences of Oracle database upgrade from 12.2 to 19.16 version.

Actually I found this error trying to figure out the absence of SQL tuning set, generated during capture system, on the replay system. It was signaled in the alert.log during executing DBMS_WORKLOAD_CAPTURE.IMPORT_AWR procedure. This led to inability of generating 'compare sqlsets report' after replaying the captured load.

Moving on to searching the cause, I found out the dump wcr_ca_sts.dmp
 
file of STS in the <replay_directory>/cap directory. Inspecting it with help if impdp (using parameters SQLFILE and MASTER_ONLY=true), it turned out that dump file contained the only table "SYSTEM"."WRRSTSTAB", which was a stage table for STS from the capture side. And this table existed in the replay database already (created after executing IMPORT_AWR procedure).

Next I tried to unpack stage table and assign STS to different owner :

SQL> begin dbms_sqltune.unpack_stgtab_sqlset (sqlset_name => '%', sqlset_owner => 'NEW_OWNER', replace => false, staging_table_name => 'WRRSTSTAB', staging_schema_owner => 'SYSTEM') ; end ;
/

It generated the error 

ORA-19377: no "SQL Tuning Set" with name like "%" exists for owner like "NEW_OWNER"
ORA-06512: at "SYS.DBMS_SYS_ERROR", line 95
ORA-06512: at "SYS.DBMS_SQLTUNE", line 10526
ORA-06512: at line 1
19377. 00000 -  "no \"SQL Tuning Set\" with name like \"%s\" exists for owner like \"%s\""
*Cause:    The user specified a filter to a pack/unpack function for the SQL
           Tuning Set that targets no STS in the SYS schema or the staging
           table, respectively
*Action:   Provide a different filter after checking the state of the system

I didn't try to change sqlset_name, I changed sqlset_owner instead to owner from the source database (SYS). After that execution of unpack_stgtab_sqlset procedure was succeeded.

At that point the last step was :

SQL> update wrr$_captures
    set sqlset_owner = 'SYS', sqlset_name = 'sqlset_name' where id = <capture_id> ;

I queried dba_workload_replays view to confirm changes from previous update. Then I was able to generate the report comparing performance using SQL tuning sets from capture and replay processes :

set pages 100 lines 30000 serveroutput on long 2000000000 longchunksize 99999 trim on trimspool on termout off head off

var sql_r clob  

spool replay_compare_sqlset_report.html

--alter session set events '19119 trace name context forever, level 0x400000';

declare
r clob ;
a varchar2 (20) ;
begin
 r := 'test string' ;
 a := dbms_workload_replay.compare_sqlset_report (replay_id1 => &1, replay_id2 => null, format => 'HTML', result => :sql_r) ;
--  dbms_output.put_line (r) ;
--  :sql_r := r ;
end ;
/

print :sql_r

spool off

exit

Good Luck !

Thursday, September 15, 2022

datapatch throws Error: prereq checks failed! patch 32218454: Error reading descriptor from registry: Couldn't open encmap cp866.enc: No such file or directory

I caught this error when I was running $ORACLE_HOME/OPatch/datapatch -verbose command to apply all the patches of 19.14 RU to 19.13 CDB and its PDBs. The error looked like this :

$ $ORACLE_HOME/OPatch/datapatch -verbose
SQL Patching tool version 19.14.0.0.0 Production on Wed Sep 14 20:31:36 2022
Copyright (c) 2012, 2021, Oracle.  All rights reserved.

Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_29306_2022_09_14_20_31_36/sqlpatch_invocation.log

Connecting to database...OK
Gathering database info...done

Note:  Datapatch will only apply or rollback SQL fixes for PDBs
      that are in an open state, no patches will be applied to closed PDBs.
      Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
      (Doc ID 1585822.1)

Bootstrapping registry and package to current versions...done
Determining current state...done

...


 PDB PDB2:
   Applied 19.13.0.0.0 Release_Update 211010063823 successfully on 16-NOV-21 07.25.29.732858 PM

Error: prereq checks failed!
 patch 32218454: Error reading descriptor from registry: Couldn't open encmap cp866.enc:
No such file or directory
at /u01/app/oracle/product/19/db_1914/perl/lib/site_perl/5.28.1/sun4-solaris-thread-multi-64/XML/Parser.pm line 187.
XML::Simple called at /u01/app/oracle/product/19/db_1914/sqlpatch/sqlpatch.pm line 8123.

Prereq check failed, exiting without installing any patches.

Please refer to MOS Note 1609718.1 and/or the invocation log
/u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_29306_2022_09_14_20_31_36/sqlpatch_invocation.log
for information on how to resolve the above errors.

SQL Patching tool complete on Wed Sep 14 20:32:23 2022

I can only guess why this error happened. Recently I plugged PDB with codepage RU8PC866 over the network and deleted it after a while. So some information had left inside registry$sqlpatch, and I can assume, it has xml part presented in ru8pc866 codepage. The datapatch invokes perl. The information is parsed by xml-parser perl module (from ORACLE_HOME). It seemed that possible solution was rebuilding registry$sqlpatch (required downtime, sounacceptable) or trying to find missed file (I chose the second). 

So, perl module xml-parser has got the following supported (from the box) codepages :

$ l $ORACLE_HOME/perl/lib/site_perl/5.28.1/sun4-solaris-thread-multi-64/XML/Parser/Encodings/
total 554
drwxr-xr-x   2 oracle   oinstall      26 Sep 14 23:01 .
drwxr-xr-x   4 oracle   oinstall       6 Mar 10  2022 ..
-rwxr-x---   1 oracle   oinstall   40706 Mar 26  2019 big5.enc
-rwxr-x---   1 oracle   oinstall   45802 Mar 26  2019 euc-kr.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 ibm866.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 iso-8859-2.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 iso-8859-3.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 iso-8859-4.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 iso-8859-5.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 iso-8859-7.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 iso-8859-8.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 iso-8859-9.enc
-rwxr-x---   1 oracle   oinstall    4821 Mar 26  2019 Japanese_Encodings.msg
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 koi8-r.enc
-rwxr-x---   1 oracle   oinstall    1950 Mar 26  2019 README
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 windows-1250.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 windows-1251.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 windows-1252.enc
-rwxr-x---   1 oracle   oinstall    1072 Mar 26  2019 windows-1255.enc
-rwxr-x---   1 oracle   oinstall   37890 Mar 26  2019 x-euc-jp-jisx0221.enc
-rwxr-x---   1 oracle   oinstall   37890 Mar 26  2019 x-euc-jp-unicode.enc
-rwxr-x---   1 oracle   oinstall   20368 Mar 26  2019 x-sjis-cp932.enc
-rwxr-x---   1 oracle   oinstall   18202 Mar 26  2019 x-sjis-jdk117.enc
-rwxr-x---   1 oracle   oinstall   18202 Mar 26  2019 x-sjis-jisx0221.enc
-rwxr-x---   1 oracle   oinstall   18202 Mar 26  2019 x-sjis-unicode.enc

I needed to find cp866.enc file somewhere. The README file says :

This directory contains binary encoding maps for some selected encodings.
If they are placed in a directory listed in @XML::Parser::Expat::Encoding_Path,
then they are automatically loaded by the XML::Parser::Expat::load_encoding
function as needed. Otherwise you may load what you need directly by
explicitly calling this function.

These maps were generated by a perl script that comes with the module
XML::Encoding, compile_encoding, from XML formatted encoding maps that
are distributed with that module. These XML encoding maps were generated
in turn with a different script, domap, from mapping information contained
on the Unicode version 2.0 CD-ROM. This CD-ROM comes with the Unicode
Standard reference manual and can be ordered from the Unicode Consortium
at http://www.unicode.org. The identical information is available on the
internet at ftp://ftp.unicode.org/Public/MAPPINGS.

So the task is to find both the scripts and get the source of 866 codepage (found CP866.TXT on ftp://ftp.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/PC)

Both compile_encoding and domap (actually, make_encmap) scripts I found in libxml-encoding-perl_2.09-1_all.deb (untar data.tar.xz from downloaded file). So, do these steps :

tar xf ./data.tar.xz 

cd usr/bin

./make_encmap cp866 /temp/service_request/perl/CP866.TXT > cp866_make.out

Edit cp866_make.out (it's actually an xml description of codepage 866) to add expat='yes' at the first line and compile :

<encmap name='cp866' expat='yes'>

./compile_encoding -o cp866.enc cp866_make.out

Put the cp866.enc file into perl module's xml-parser encodings directory, for me it was /u01/app/oracle/product/19/db_1914/perl/lib/site_perl/5.28.1/sun4-solaris-thread-multi-64/XML/Parser/Encodings

and rerun datapatch. In my particular case I cought another error not related to this topic :

Unsupported named object type for bind parameter at /u01/app/oracle/product/19/db_1914/sqlpatch/sqlpatch.pm line 5783.

The cause of it was an absense of tempfile inside one of PDB's temporary tablespace. After adding I ended up with whole CDB patched.

The conclusion : do not use stale and deprecated codepages to keep your data in 😀 ! Oracle recommends all its customers to migrate to AL32UTF8 (but it requres much efforst in many cases and it depends, actually...)

PS. I still don't understand how the patch numbered 32218454 (database 19.10 RU) has turned out to be a cause of this issue. Only one assumption : it had been used with one of PDBs ORACLE_HOME before being plugged in into CDB.



Sunday, September 4, 2022

When crontab schedule is not enough

As far as you know, cron provides the schedule on the following time attributes: minute, hour, day of the week, month, day of month. What if you need to set the specific schedule, for example, you need (not) to fire the event only on last working day of the current month and on the next day after it (2 days in a row) ? 

The application is restarting every day. Let's assume that working days between Monday and Friday, the current month (September) has 30 days, and last working day is Friday, 28th. So, we need (not) to fire the event on 28th and 29th. How to do this using only crontab file ? Have a look :

#
# If END_OF_MONTH=true, the app server is not stopped and not restarted automatically.
# END_OF_MONTH is set to true when current day is last working day of month, and next day after it (2 days in row).
# a and b - intermediate variables to keep END_OF_MONTH intermediate value.
# Exception when 30.04 is Monday (as in 2018 and 2029; then end of month must be moved to Saturday, 28.04, because 01.05 (day off) is Tuesday),
# or 29.04 is Radonica (in 2025 and 2036, so 30.04 is moved to 03.05, end of month in this case 28.04).
#
cy='date +%Y'
cm='LANG=en date +%b'
nm1wd='$(test $(eval $cm) = "Dec") && eval $(echo "LANG=en date +%u -d \"$(LANG=en date +%b -d "next month") 1 +1year\"") || eval $(echo "LANG=en date +%u -d \"$(LANG=en date +%b -d "next m
onth") 1\"")'
lcmd='LANG=en date +%d -d "$(LANG=en date +%b -d "next month") 1 -1day"'
cday='date +%-d'
cwday='date +%u'
fr1="cal -m | tail -n +3 | awk '{print $5}' | sed -n '4p'"
fr2="cal -m | tail -n +3 | awk '{print $5}' | sed -n '5p'"
last_fr='test -z "$(eval $fr2)" && echo $(eval $fr1) || echo $(eval $fr2)'
sa1="cal -m | tail -n +3 | awk '{print $6}' | sed -n '4p'"
sa2="cal -m | tail -n +3 | awk '{print $6}' | sed -n '5p'"
last_sa='test -z "$(eval $sa2)" && echo $(eval $sa1) || echo $(eval $sa2)'
_END_OF_MONTH='$(test $(eval $nm1wd) -ge 3 -a $(eval $nm1wd) -le 5 -a $(eval $cday) -eq $(eval $lcmd) || (test $(eval $nm1wd) -eq 6 -a $(eval $cday) -eq $(eval $last_fr)) || (test $(eval $n
m1wd) -eq 7 -o $(eval $nm1wd) -eq 1 && test $(eval $cday) -eq $(eval $last_fr) -o $(eval $cday) -eq $(eval $last_sa)) || test $(eval $nm1wd) -eq 2 -a $(eval $cday) -eq $(eval $last_sa) || t
est $(eval $cday) -eq 1 -a $(eval $cwday) -ge 2 -a $(eval $cwday) -le 6) && echo true'
apr_d='LANG=en date +%d%b%a'
apr_eom='$(test $(eval $apr_d) = "28AprSat" -o $(eval $apr_d) = "29AprSun" -o $(eval $apr_d) = "28AprMon" -o $(eval $apr_d) = "29AprTue") && echo true || echo false'
apr_no_eom='$(test $(eval $apr_d) = "30AprMon" -o $(eval $apr_d) = "01MayTue" -o $(eval $apr_d) = "30AprWed" -o $(eval $apr_d) = "01MayThu") && echo true || echo false'
END_OF_MONTH='$(test $(eval $apr_eom) = "true" -a $(eval $cy) != "2031" ) && echo true || echo $(eval $_END_OF_MONTH)'
END_OF_MONTH='$(test $(eval $apr_no_eom) = "true" -a $(eval $cy) != "2031" ) && echo false || echo $(eval $_END_OF_MONTH)'
END_OF_MONTH='$(test \( $(eval $apr_d) = "27AprSat" -o $(eval $apr_d) = "28AprSun" \) -a $(eval $cy) = "2041") && echo true || echo $(eval $_END_OF_MONTH)'
END_OF_MONTH='$(test \( $(eval $apr_d) = "30AprTue" -o $(eval $apr_d) = "01MayWed" \) -a $(eval $cy) = "2041") && echo false || echo $(eval $_END_OF_MONTH)'
 

00    02    *   *     *           [ "$(echo $(eval $END_OF_MONTH))" != "true" ] && su - app_user -c "/home/app_user/app stop"

00    03    *   *     1-7         [ "$(echo $(eval $END_OF_MONTH))" != "true" ] && su - app_user -c "/home/app_user/app restart"

The logic is build around using bash eval and test buildin commands. If END_OF_MONTH is set to true, then nothing happens, app stop and restart tasks aren't run. There are couple of exceptions regarding to my country : 1st of May and Cristian Orthodox holiday named Radonica are always day off; because of this the movements of day off and working days are possible. I tried to count these exceptions in crontab too.

Good Luck !


Wednesday, August 31, 2022

dnf (yum) throws "The SSL certificate failed verification" error

All of a sudden an application named dnf (a successor of yum, used as package manager on Oracle Linux OS) stopped functioning, but throwing the message from the title, even when it's being called with help option(s). The main cause was local machine time, it wasn't synchronized with real clock time. The solution was to sync it, as well as modify the chrony.conf (add string like 'server vm_host iburst' in order to sync time quickly after save_vm->restore_vm operation) and run the dnf with required options :

# systemctl stop chronyd

# chronyd -q 'pool pool.ntp.org iburst'

# systemctl start chronyd

# dnf help

Good Luck !

Monday, August 29, 2022

Authentication of Oracle database users in Microsoft Active Directory - outlook

The assignment - to exclude maintenance of two equal user entities (in Microsoft AD and Oracle DB) and provide password-less entrance into database provided successful authorization in AD. User starts its session on Windows, and run 2-level applications which require authorization in the databases as well.

All the info below is a conspectus. Please look through official Oracle Security documents and blogs for the information in detail. Good starting point is :

https://blogs.oracle.com/database/post/make-someone-else-do-the-work-managing-oracle-database-19c-users-in-active-directory-part-1-kerberos 

1. Important database parameter is os_authent_prefix. Default value of it is "OPS$". It's a prefix to the operating system (OS) username. In order to have equal database and OS usernames, leave the null value for this parameter. It's not online and PDB modifiable - you need to restart the DB/CDB. 

2. Authentication goes through Kerberos5 protocol, port 88 (default) should be accessible from database server(s).

3. Edition of database - Enterprise (EE) 

4. You need to create so called proxy technological AD user on behalf of whum the kerberos5 ticket will be received on the database side. Toggle "Password never expires", deselect "DES encryption" and "Kerberos Pre-authentication".

On AD side enter the command :

c:> ktpass -princ oracle/db_server_dns_name.dns_domain_name@DOMAIN_NAME -pass <password> -mapuser technological_user@DOMAIN_NAME -crypto all -ptype KRB5_NT_PRINCIPAL -out c:\krb5.keytab. 

DOMAIN_NAME (i.e. AD domain name) must always be in upper-case, including in database sql statements containing oracle usernames.

Put resulted file to $ORACLE_HOME/network/admin server directory.

!Important! Objects in AD have so called knum attribute (analog of SCN 😀 in Oracle DB), which goes forward after any alterations of the user. So keep the keytab file in sync with this AD user. After any alteration of technological user recreate keytab file, because database checks the equality of technological user's knum and knum inside keytab. Also, because of this, create dedicated AD user for every other (standby or else) database servers. 

5. Example of Oracle server configuration files :

5.1 Add these lines into sqlnet.ora

# kerberos parameters
sqlnet.kerberos5_conf                  = /u01/app/oracle/product/1910/network/admin/krb5.conf
sqlnet.kerberos5_conf_mit              = true
sqlnet.fallback_authentication         = true
sqlnet.authentication_kerberos5_service = oracle
sqlnet.authentication_services         = (beq, kerberos5pre, kerberos5)
names.directory_path                   = (ldap, tnsnames, ezconnect, hostname)
# server-side only
sqlnet.kerberos5_keytab                = /u01/app/oracle/product/1910/network/admin/krb5.keytab

5.2 krb5.conf

[libdefaults]
 default_realm = DOMAIN_NAME
 clockskew = 6000
 passwd_check_s_address = false
 noaddresses = true
 forwardable = yes
[realms]
  DOMAIN_NAME = {
   kdc = domain_controller_1.DOMAIN_NAME:88
   kdc = domain_controller_2.DOMAIN_NAME:88
 }
[domain_realm]
  .domain_name =
DOMAIN_NAME 
   domain_name =
DOMAIN_NAME
  .
DOMAIN_NAME = DOMAIN_NAME
  
DOMAIN_NAME = DOMAIN_NAME

5.3 Also it's important to enable tracing on the server in case of  troubleshooting (there gonna be lots of trace files with huge amount of space; so keep an eye on the free space in trace file directory). The example of enabled tracing in sqlnet.ora :

# trace
DIAG_ADR_ENABLED = off
TRACE_LEVEL_SERVER = on
TRACE_DIRECTORY_SERVER = /u01/app/oracle/product/1910/krb5/trace
TRACE_FILE_SERVER=oracle

Some of the settings are enabled on the fly, but very likely you should restart database to enable most of them.

6. On the client machine add authentication settings to sqlnet.ora :

sqlnet.kerberos5_conf                  = %ORACLE_HOME%\network\admin\krb5.conf
sqlnet.kerberos5_conf_mit              = true
sqlnet.fallback_authentication         = true
sqlnet.authentication_kerberos5_service = oracle
sqlnet.authentication_services         = (kerberos5pre, kerberos5)
sqlnet.kerberos5_cc_name                = OSMSFT://

krb5.conf may be replicated from the database server. 

!Important! If the task or program doesn't support AD authentication and there's a need to logon to database user authenticated by the database, use another dedicated sqlnet.ora with sqlnet.authentication_services = none.

Also make sure that file %windir%\system32\drivers\etc\services contains lines:

kerberos 88/tcp kerberos5 krb5 kerberos-sec #Kerberos

kerberos 88/udp kerberos5 krb5 kerberos-sec #Kerberos

7. Alter database user to authenticate using Microsoft AD :

SQL> alter user username identified externally as '<ad_user>.DOMAIN_NAME' ;

8. Check username and corresponding AD usernames (with domain) in rhw :

SQL> select username, external_name from dba_users where external_name is not null ;

9. Connect to the database without entering username/password, for example :

c:> sqlplus /@tns_name_for_connect

SQL> select user, sys_context('userenv','authentication_method'), sys_context('userenv','authenticated_identity') from dual ;

Basically, that's all. Hope it'll help ! Enjoy ;-)

Thursday, August 25, 2022

ORA-06502: PL/SQL: numeric or value error when calling PL/SQL procedure via dblink

Suppose we've got 2 Oracle databases connected via db link : client db and server db. On the client database the PL/SQL procedure (from the package) calls another procedure via database link (from package on the server database) and saves the result into a cursor (which is also declared via the same database link). Next the result is processed and data is delivered to the application.

Out of the sudden, an application started to get the error 6502 sometimes. As it's figured out the cause laid in the length of varchar2 variable (let's call it 'a'). So, 'a' is declared varchar2 (100) variable (in bytes) inside a pl/sql procedure on the client side. It gets data via database link from the cursor (declared via database link as I said before), created on the server side (the corresponding column also has varchar2 (100) datatype (in bytes)). 

It seems when client database gets the value of the this column in language different from English it internally converts it into client database characterset (from server database characterset - CL8MSWIN1251 to AL32UTF8 - characterset of client database). When converted value's length is greater then 100 bytes - the error ORA-06502 is arising. At this point increasing variable length (to 1000) fixed the issue. 

I hope you were able to comprehend the above ;-) Good Luck !!!

PS: simplified version of the client procedure :

PACKAGE BODY package_body AS
c_num varchar2(30);
FUNCTION getfullinfo(c VARCHAR2,date_from VARCHAR2,date_to VARCHAR2,flag NUMBER DEFAULT 0, cr NUMBER DEFAULT 0, megat NUMBER DEFAULT 0) RETURN curstype
IS
curs cursType;
recTableremote server_user.server_package.table_type@db_link;
recTablelocal local_user.local_package.table_type;
rec
local_user.local_package.record_type;
BEGIN

recTablelocal := {initializing the type} ;
server_user.server_package.procedure@db_link(c, date_from, date_to, flag, cr, megat, recTableremote);
 

for i in 1 .. recTableremote.count
loop
rec.row_numb :=
recTableremote(i).row_numb;
...



Friday, June 24, 2022

CRSD fails to start with ORA-01017 invalid username/password

The grid alert.log says smth like this :

 2022-06-24 09:01:27.377 [OHASD(34093)]CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'hostname'
2022-06-24 09:01:27.835 [CRSD(30213)]CRS-1013: The OCR location in an ASM disk group is inaccessible. Details in /u01/oracle/diag/crs/hostname/crs/trace/crsd.trc.
2022-06-24 09:01:27.842 [CRSD(30213)]CRS-0804: Cluster Ready Service aborted due to Oracle Cluster Registry error [PROC-26: Error while accessing the physical storage Storage layer error [I
nsufficient quorum to open OCR devices] [0]]. Details at (:CRSD00111:) in /u01/oracle/diag/crs/hostname/crs/trace/crsd.trc

crsd.trc says something like this :

 2022-06-24 09:01:27.834 :  OCRRAW:1: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied


2022-06-24 09:01:27.834*:kgfo.c@1014: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied

2022-06-24 09:01:27.834 :  OCRRAW:1: -- trace dump on error exit --

2022-06-24 09:01:27.834 :  OCRRAW:1: Error [kgfoAl06] in [kgfokge] at kgfo.c:3182

2022-06-24 09:01:27.834 :  OCRRAW:1: ORA-01017: invalid username/password; logon denied


2022-06-24 09:01:27.834 :  OCRRAW:1: Category: 7  

2022-06-24 09:01:27.834 :  OCRRAW:1: DepInfo: 1017

2022-06-24 09:01:27.834 :  OCRRAW:1: -- trace dump end --

 OCRASM:1: SLOS : SLOS: cat=7, opn=kgfoAl06, dep=1017, loc=kgfokge

2022-06-24 09:01:27.834 :  OCRASM:1: ASM Error Stack : ORA-01017: invalid username/password; logon denied

2022-06-24 09:01:27.835 :  OCRASM:1: proprasmo: kgfoCheckMount returned [7]
2022-06-24 09:01:27.835 :  OCRASM:1: proprasmo: The ASM instance is down
2022-06-24 09:01:27.835 :  OCRRAW:1: proprioo: Failed to open [+DATAC6/cl_name/OCRFILE/registry.255.1081797515]. Returned proprasmo() with [26]. Marking location as UNAVAILABLE.
2022-06-24 09:01:27.835 :  OCRRAW:1: proprioo: No OCR/OLR devices are usable
 OCRUTL:1: u_fill_errorbuf: Error Info : [Insufficient quorum to open OCR devices]
default:1: u_set_gbl_comp_error: comptype '107' : error '0'
2022-06-24 09:01:27.836 : default:1: clsvactversion:4: Retrieving Active Version from local storage.
2022-06-24 09:01:27.838 : CSSCLNT:1: clssgsgrppubdata: group (ocr_cl_name) not found
2022-06-24 09:01:27.838 :  OCRRAW:1: proprio_repairconf: Failed to retrieve the group public data. CSS ret code [20]
2022-06-24 09:01:27.838 :  OCRRAW:1: proprioo: Failed to auto repair the OCR configuration.
2022-06-24 09:01:27.838 :  OCRRAW:1: proprinit: Could not open raw device
2022-06-24 09:01:27.841 :  OCRAPI:1: a_init:16!: Backend init unsuccessful : [26]
2022-06-24 09:01:27.841 :  OCRAPI:1: estack 'PROC-00026: Error while accessing the physical storage'
2022-06-24 09:01:27.842 :  CRSOCR:1: [    ERROR]  OCR context init failure.  Error: PROC-26: Error while accessing the physical storage Storage layer error [Insufficient quorum to open OCR
devices] [0]
2022-06-24 09:01:27.843 :    CRSD:1: [     NONE]  Created alert : (:CRSD00111:) :  Could not init OCR, error: PROC-26: Error while accessing the physical storage Storage layer error [Insuff
icient quorum to open OCR devices] [0]
2022-06-24 09:01:27.843 :    CRSD:1: [    ERROR] [PANIC] CRSD exiting: Could not init OCR, code: 26
2022-06-24 09:01:27.843 :    CRSD:1: [     INFO]  Done.

2022-06-24 09:01:27.834 :  OCRRAW:1: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied


2022-06-24 09:01:27.834*:kgfo.c@1014: kgfo_kge2slos error stack at kgfoAl06: ORA-01017: invalid username/password; logon denied

2022-06-24 09:01:27.834 :  OCRRAW:1: -- trace dump on error exit --

2022-06-24 09:01:27.834 :  OCRRAW:1: Error [kgfoAl06] in [kgfokge] at kgfo.c:3182

2022-06-24 09:01:27.834 :  OCRRAW:1: ORA-01017: invalid username/password; logon denied


2022-06-24 09:01:27.834 :  OCRRAW:1: Category: 7  

2022-06-24 09:01:27.834 :  OCRRAW:1: DepInfo: 1017

2022-06-24 09:01:27.834 :  OCRRAW:1: -- trace dump end --

 OCRASM:1: SLOS : SLOS: cat=7, opn=kgfoAl06, dep=1017, loc=kgfokge

2022-06-24 09:01:27.834 :  OCRASM:1: ASM Error Stack : ORA-01017: invalid username/password; logon denied

2022-06-24 09:01:27.835 :  OCRASM:1: proprasmo: kgfoCheckMount returned [7]
2022-06-24 09:01:27.835 :  OCRASM:1: proprasmo: The ASM instance is down


The cause of this has been not corrupted ASM pwfile or something like that. It was much more easier - accidentally changed default group id of grid software owner. Instead of 

% id -a oracle
uid=1000(oracle) gid=1001(oinstall) groups=1001(oinstall),1002(dba)

it was set to

% id -a oracle
uid=1000(oracle) gid=1002(dba) groups=1001(oinstall),1002(dba)

Fixing it via usermod and restarting the server resolved the issue 😀


Wednesday, June 22, 2022

[FATAL] Error in invoking target 'irman ioracle...' of makefile 'ORACLE_HOME/rdbms/lib/ins_rdbms.mk'. See 'ORA_INVENTORY/logs/InstallActions.../installActions....log' for details

This error occurred during installation of new Oracle RU to new oracle home. The linking phase was going on. The out-of-memory operating system protection mechanism had killed the linking process. 

The main cause of it had been vm.swappiness=0 settings (had been set in sysctl.conf before); in other words, operating system was disabling any swap activity if it could do so. Any values more that 0 resolved the issue (by repeating the installation).

Friday, June 3, 2022

Exception Occured while executiong Apply Operation: ApplySession failed in system modification phase... 'Execution of PRE script failed, with return value = 127'

Today I got the following error installing the RU7 for OEM 13.5 :

$ /u01/app/oem13c/emcc_135/OMSPatcher/omspatcher apply -property_file /u01/app/oem13c/aux/property_file

OMSPatcher Automation Tool
Copyright (c) 2017, Oracle Corporation.  All rights reserved. 

...

OMSPatcher failed to apply following patch(es) "34003606" "34003614" "34003617" "34003654" "34003678"  to core/plugin Oracle home(s).


Logfile said that the error occured while executing PRE script :

[Jun 3, 2022 10:44:33 AM]    NApplyReport Log file: /u01/app/oem13c/emcc_135/cfgtoollogs/opatch/opatch2022-06-03_10-41-10AM_1.log
[Jun 3, 2022 10:45:30 AM]    Exception Occured while executiong Apply Operation: ApplySession failed in system modification phase... 'Execution of PRE script failed, with return value = 127
'
[Jun 3, 2022 10:45:30 AM]    NApply Logfile: /u01/app/oem13c/emcc_135/cfgtoollogs/opatch/opatch2022-06-03_10-41-10AM_1.log

The main cause of that was the spaces in the path of unpacked RU patch directory. But what to do in this case ? Trying to resume (or reapply) the installation is obvious, but it wouldn't work because of the spaces in the path neither omspatcher rollback would.

But how to overcome spaces-in-the-path issue ? Go to the OMS oracle home and fix it manually 😉 :

1. unzip/move the patch to the new location without spaces in the path.

2. edit some files in the OMS oracle home

$ cd $ORACLE_HOME/.omspatcher_storage

Edit the files :

- .patch_failure.properties

- restoreSteps.txt

- file under the execution_txt variable inside .patch_failure.properties

3. run omspatcher resume command and Bob is your uncle !

Good Luck !

Thursday, May 26, 2022

java.sql.SQLException: Non supported character set (add orai18n.jar in your classpath) - Oracle Enterprise Manager 13.5 upgrade/install

The repository of EM Cloud Control was historically situated in the database of different from AL32UTF8 characterset. The upgrade procedure of EM is stucking into the error in the title. Had the character set of repository database been AL32UTF8, this error would have never been happen (but I found this is a tricky way because encrypted EM key is kept in the repository and it depends on the id (guid ?) of database as well).

The following steps was made to resolve.

1. Unzip em13500_linux64.bin into temporary catalog

% unzip -q /u02/install/13.5/em13500_linux64.bin -d /u02/install/13.5/unzip/

2. Unzip EM installer, put the file orai18n.jar from existing 19c oracle home and make some modifications :

% cd /u02/install/13.5/unzip/Disk1/stage/Components/oracle.sysman.em.installer/13.5.0.0.0/DataFiles

% mkdir unzip_fg1

% unzip -q filegroup1.jar -d unzip_fg1

% cd unzip_fg1/oui/em

% cp $ORACLE_HOME/jlib/orai18n.jar .

% cp -p ../../../compDef.xml ../../../compDef.xml.orig

% vi ../../../compDef.xml 

Put the line describing orai18n.jar into compDef.xml

...

             <file size="1661488" dest="orai18n.jar"/>
...

% cd ../.. ; pwd

/u02/install/13.5/unzip/Disk1/stage/Components/oracle.sysman.em.installer/13.5.0.0.0/DataFiles/unzip_fg1

% cp -p ../filegroup1.jar ../filegroup1.jar.orig

% zip -r ../filegroup1.jar *

% cd /u02/install/13.5/unzip/Disk1/install

% cp -p oraparam.ini oraparam.ini.orig

% cp -p linux64/oraparam.ini linux64/oraparam.ini.orig

Modify oraparam.ini to include the new jar file in the classpath :

...

EXT_CLASSPATH=oui/em/orai18n.jar,...

...

% cd /u02/install/13.5

% cp -p em13500_linux64.bin em13500_linux64.bin.orig

% cd unzip

% zip -u ../em13500_linux64.bin Disk1/stage/Components/oracle.sysman.em.installer/13.5.0.0.0/compDef.xml

% zip -u ../em13500_linux64.bin Disk1/stage/Components/oracle.sysman.em.installer/13.5.0.0.0/DataFiles/filegroup1.jar

% zip -u ../em13500_linux64.bin Disk1/install/oraparam.ini

% zip -u ../em13500_linux64.bin Disk1/install/linux64/oraparam.ini

3. That's it ! Run the installation or upgrade 😀


Wednesday, May 25, 2022

ORA-29548: Java system class reported: release of Java system classes in the database... - datapatch - rollback OJVM patches on PDB

I created PDB (version 19.10) via database link (from PDB version 19.13).

I opened it - and there were a lot of violations in PDB opening (patch mismatches etc.)

I run datapatch -verbose -pdb ... to rollback all the patches from 19.13 and implement patches from 19.10 home.

The output was :

Validating logfiles...done                                                                     
Patch 33192694 rollback (pdb AAA): WITH ERRORS                
 logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33192694/24442454/33192694_rollback_CDBR_AAA_2022May25_15_18_42.log (errors)
 -> Error at line 55: script javavm/install/jvmpsupdi.sql         
     - SP2-0310: unable to open file "/u01/app/oracle/product/db_1910/javavm/install/jvmpsupdii.sql"
Patch 29213893 rollback (pdb AAA): SUCCESS                                
 logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/29213893/24384541/29213893_rollback_CDBR_AAA_2022May25_15_18_43.log (no errors)
Patch 33430073 rollback (pdb AAA): WITH ERRORS                
 logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33430073/24509446/33430073_rollback_CDBR_AAA_2022May25_15_18_46.log (errors)
 -> Error at line 106017: script rdbms/admin/dpload.sql                  
     - ORA-65047: Object SYS.KU$_COLLIST_VIEW is invalid or compiled with errors in root.
     - ORA-06512: at "SYS.RECOMP_CATMETA_VIEWS", line 35          
     - ORA-06512: at line 8                                                                                                                                                                  
Patch 33192793 rollback (pdb AAA): WITH ERRORS               
 logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33192793/24467360/33192793_rollback_CDBR_AAA_2022May25_15_29_03.log (errors)
 -> Error at line 217991: script md/admin/catsem.sql                      
     - ORA-29548: Java system class reported: release of Java system classes in the
     - database (19.0.0.0.211019 1.8) does not match that of the oracle executable
     - (19.0.0.0.0 1.8)                                                                         
     - ORA-06512: at "SYS.DBMS_JAVA", line 587                                                                                                                                               
     - ORA-06512: at line 1         



According to the logs, the real cause laid in the absence of $ORACLE_HOME/javavm/install/mpsupdii.sql file, which was in this place in oracle home version 19.13, but it also was in rollback catalog related to rollbacking patch :

 gfind -name "*jvmpsupdii*"
./sqlpatch/33192694/24442454/rollback_files/19.1.0.0.0/javavm/install/jvmpsupdii.sql

 I just put this file to directory ORACLE_HOME/javavm/install and rerun datapatch without errors. Good Luck !


Wednesday, May 11, 2022

In-place upgrade of Oracle Linux 6 or 7 to Oracle Linux 8

OL upgrade from version 6 to 8 requires upgrade version 6 to version 7 first, then you can upgrade version 7 to 8. Next steps describe briefly how to upgrade OL 6.10 (UNL-registered) to 8.5. It doesn't replace the documentation. Before you start, consider the restoration procedure, the upgrade might possibly fail.

1. Enable additional repositories.

Install additional package end enable new repository system :

# yum install oracle-softwarecollection-release-el6

# /usr/bin/ol_yum_configure.sh

I enabled all disabled repositores, but it's not required. 

# yum-config-manager --enable \*

If you receive the error :

 Traceback (most recent call last):
 File "/usr/bin/yum-config-manager", line 192, in <module>
   repo.cfg.options, repo.iteritems, repo.optionobj,
AttributeError: 'RhnRepo' object has no attribute 'cfg'

then disable rhn plugin in file /etc/yum/pluginconf.d/rhnplugin.conf.

Check enabled repositories :

# yum repolist

2. Update packages of Linux 6 to the last available versions.

# yum update

You may want to modify proxy settings (/etc/sysconfig/rhn/up2date) or resolve dependency issues. Reboot the system. You might also disable any network mounted filesystems in /etc/fstab. Change mount options for the local file system to defaults, just to be on the safe side.

3. Install additional packages required for upgrade to OL 7 and check possible issues before upgrade :

# yum install openscap redhat-upgrade-tool preupgrade-assistant preupgrade-assistant-el6toel7 preupgrade-assistant-el6toel7-data-0 preupgrade-assistant-tools preupgrade-assistant-ui

# preupg

4. Copy iso of OL7.6 into local disk, mount it and run upgrade to 7.6. redhat-upgrade-tool-cli works only with OL7.6 iso, so you have to upgrade to 7.6 first, then upgrade to 7.9 (the last release of OL7) aftermath. Copy the content of iso into local directory, you can use rsync -av for it. Open session on the server console, do not do the upgrade over the network.

# mount -o loop /install/OracleLinux-R7-U6-Server-x86_64-dvd.iso /mnt/temp/

# mkdir /install/ol7r6 ; cd /mnt/temp ; rsync -av ./* /install/ol7r6

# redhat-upgrade-tool-cli --force --iso=/install/OracleLinux-R7-U6-Server-x86_64-dvd.iso 

--debuglog=/tmp/upgrade.log --cleanup-post

I used --force option on my own risk. I had only two failed prerequisites related to GNOME and KDE desktop environment, so I decided to ignore them forcibly. 

I had to reboot the server at this stage also because upgrade procedure gave the error when it restarted the network service as preliminary action. The upgrade changes network card names from ethX to netX after reboot, but it actually used new names before (mandatory before upgrade) reboot. After running again redhat-upgrade-tool-cli, when script's run successfully, go further :

                                                          [  OK  ]
Finished. Reboot to start upgrade.

5. Reboot to start upgrade to 7.6

At this phase system boots into temporary upgrade shell and upgrade is being performed, and system is rebooted again into new kernel.

6. Upgrade GRUB boot loader to GRUB2.

Please use documentation link to do : upgrade_grub_to_grub2

7. Install last updates (patches) to 7.9 release and reboot :

# yum-config-manager --enable \*

# yum upgrade

# reboot

8. Unregister the system from Unbreakable Linux Network (ULN, if registered)  and reboot.

9. Add leapp_ol7 repository and install leapp utility (or modify oracle-linux-ol7-.repo.rpmnew to enable ol7_leapp repository) :

# reboot

# cat /etc/yum.repos.d/leapp.repo
[ol7_leapp]
name=Leapp Upgrade Utilities for Oracle Linux $releasever ($basearch)
baseurl=https://yum.oracle.com/repo/OracleLinux/OL7/leapp/x86_64/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
 

# yum install -y leapp --enablerepo=ol7_leapp,ol7_latest

All steps about using leapp for upgrade OL7 to OL8 described in detail in the document Performing Systems Upgrade With Leapp.

10. Grant root login over ssh explicitly (PermitRoolLogin=yes in sshd_config) and perform preupgrade :

# leapp preupgrade --oraclelinux

11. Analyze file /var/log/leapp/leapp-report.txt for issues/errors. As for me - I received the following issues :

============================================================
                    UPGRADE INHIBITED  ============================================================                                                                                                                                  

Upgrade has been inhibited due to the following problems:
   1. Inhibitor: Confirm only RDMA UEKR6 is available on OL8.
   2. Inhibitor: Confirm kernel upgrade to UEKR6 for RDMA?
   3. Inhibitor: Missing required answers in the answer file
   4. Inhibitor: Missing required answers in the answer file
   5. Inhibitor: Missing required answers in the answer file
Consult the pre-upgrade report for details and possible remediation.

============================================================
                    UPGRADE INHIBITED
============================================================     

12. To fix inhibitor 1, I'd enabled appropriate repositories :

# yum-config-manager --enable ol8_UEKR6,ol8_UEKR6_RDMA

I'd changed link of baseurl in repository file to publicly available site yum.oracle.com :

[ol8_UEKR6]
name=Latest Unbreakable Enterprise Kernel Release 6 for Oracle Linux $releasever ($basearch)
baseurl=https://yum.oracle.com/repo/OracleLinux/OL8/UEKR6/x86_64/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

[ol8_UEKR6_RDMA]
name=Oracle Linux 8 UEK6 RDMA ($basearch)
baseurl=http://yum.oracle.com/repo/OracleLinux/OL8/UEKR6/RDMA/x86_64/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1

Then I entered answeres into answerfile and repeated preupgrade procedure :

# cat /var/log/leapp/answerfile
[confirm_RDMA_UEKR6_only_on_OL8]
# Title:              None
# Reason:             Confirmation
# =================== confirm_RDMA_UEKR6_only_on_OL8.confirm ==================
# Label:              Confirm RDMA UEKR6 on OL8 only? If no, the upgrade process will be interrupted.
# Description:        Confirm only RDMA UEKR6 is available on OL8.
# Type:               bool
# Default:            None
# Available choices: True/False
# Unanswered question. Uncomment the following line with your answer
confirm = True

[remove_pam_pkcs11_module_check]
# Title:              None
# Reason:             Confirmation
# =================== remove_pam_pkcs11_module_check.confirm ==================
# Label:              Disable pam_pkcs11 module in PAM configuration? If no, the upgrade process will be interrupted.
# Description:        PAM module pam_pkcs11 is no longer available in OL-8 since it was replaced by SSSD.
# Type:               bool
# Default:            None
# Available choices: True/False
# Unanswered question. Uncomment the following line with your answer
confirm = True

[RDMA_upgrade_kernel_to_UEKR6]
# Title:              None
# Reason:             Confirmation
# ==================== RDMA_upgrade_kernel_to_UEKR6.confirm ===================
# Label:              Confirm kernel upgrade to UEKR6 for RDMA? If no, the upgrade process will be interrupted.
# Description:        Confirm upgrade kernel to UEKR6 for RDMA.
# Type:               bool
# Default:            None
# Available choices: True/False
# Unanswered question. Uncomment the following line with your answer
confirm = True

# leapp preupgrade --oraclelinux

At this point you may want to update proxy settings as well. You should do it in several places actually (not in one place, as you possible did when upgrading to OL7) :

- /etc/yum.repos.d/leapp-upgrade-repos-ol8.repo. Here you must add proxy entries for everyone enabled OL8 repository

[ol8_UEKR6]
name=Latest Unbreakable Enterprise Kernel Release 6 for Oracle Linux $releasever ($basearch)
baseurl=https://yum$ociregion.$ocidomain/repo/OracleLinux/OL8/UEKR6/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
proxy=http://127.0.0.1:3128

[ol8_appstream]
name=Oracle Linux 8 Application Stream ($basearch)
baseurl=https://yum$ociregion.$ocidomain/repo/OracleLinux/OL8/appstream/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
proxy=http://127.0.0.1:3128

[ol8_baseos_latest]
name=Oracle Linux 8 BaseOS Latest ($basearch)
baseurl=https://yum$ociregion.$ocidomain/repo/OracleLinux/OL8/baseos/latest/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
proxy=http://127.0.0.1:3128

- /etc/dnf/dnf.conf

[main]
gpgcheck=1
installonly_limit=3
clean_requirements_on_remove=True

sslverify=False

proxy=http://127.0.0.1:3128
 

- just in case I modified /etc/yum.conf as well (proxy=...) and created some shell variables for curl, which is used for getting new versions of packages :

export all_proxy=http://127.0.0.1:3128 https_proxy=http://127.0.0.1:3128 http_proxy=http://127.0.0.1:3128

Just to point you out : you may need additional free space in /var directory (/var/lib/leapp, in my case there was only 8GB free, but leapp requested 20GB free). Solution - move leapp directory to filesystem with more free space and make a symbolic link to it.

13. After receiving "green" report, one can upgrade the OS :

Debug output written to /var/log/leapp/leapp-preupgrade.log

============================================================
                          REPORT
============================================================                                                                                                                                  

A report has been generated at /var/log/leapp/leapp-report.json
A report has been generated at /var/log/leapp/leapp-report.txt

============================================================
                      END OF REPORT
============================================================                                                                                                                                  

Answerfile has been generated at /var/log/leapp/answerfile

# leapp upgrade --oraclelinux

Upgrade report should also be "green". Reboot the system. The upgrade procedure will start immediately after running operating system (new entry is created in the boot loader configuration and is used during reboot), and OS will be rebooted again.

13. Verify the upgraded OS.

# cat /etc/*release

# uname -r

14. Register system in Unbreakable Linux Network (if needed) and install last patches. You may need to recreate grub2 configuration (in order to reorder grub menu entries - remove OL7 and add OL8 lines) by :

# grub2-mkconfig -o /boot/grub2/grub.cfg

15. Relink/reinstall/upgrade all the needed software.

16. Enjoy !

P.S. leapp software can be used for upgrade to OL9 as well.

Monday, April 25, 2022

Authorization not available. Check if polkit service is running or see debug message for more information

Another comic situation which I've trapped into recently, was the huge delay when I logged into Oracle Linux server via ssh or on the console.

Analyzing boot.msg logfile led to the message from the post's title. The cause was actually wrong actions, applied during repairing operating system from bootable iso image. After some failures before I had had no run and lock directores inside /var, only run.rpmnew and lock.rpmnew. So I renamed them to run and lock respectively, and this was the main cause of the fault. Instead it I had to create symbolic links like these following :

# cd /var

# mv run run.old ; ln -s ../run 

# mv lock lock.old ; ln -s ../run/lock

That's it ! After reboot all services worked fine, there were no delays during logons and previously mentioned messages disappeared from boot.msg logfile :-) Good Luck !

Friday, April 22, 2022

ping: Name or service not known

Recently I encountered a stupid situation on Oracle Linux 8, when I wasn't unable to connect from OL8 server to another server via its dns name. For example

# ping www.google.com
ping: www.google.com: Name or service not known

The cause was very simple : the space before word 'nameserver' in the /etc/resolv.conf file. The contents looked like :

; nameserver 192.168.56.1
 nameserver 192.168.0.1

After correction the mentioned file started to look like :

; nameserver 192.168.56.1
nameserver 192.168.0.1

And all started to work as expected :)

# ping www.google.com                  
PING www.google.com (142.250.184.196) 56(84) bytes of data.
64 bytes from fra24s11-in-f4.1e100.net (142.250.184.196): icmp_seq=1 ttl=63 time=42.4 ms
64 bytes from fra24s11-in-f4.1e100.net (142.250.184.196): icmp_seq=2 ttl=63 time=42.1 ms

Good Luck !

 

Sunday, April 17, 2022

GRUB2 doesn't load OS and drills down into BASH-like prompt

There may be many situations with unworking GRUB2 configuration, where the OS can't be loaded. In my case I made an error, migrating old GRUB into new GRUB2 configuration, and created new config file with conf extension (like in the previous GRUB :-) ), but the right extension should be cfg.

So, when GRUB2 gave me the prompt, I entered the command :

GRUB2> configfile /grub2/grub.conf

Then, when OS booted, I generate new config file using command :

# grub2-mkconfig -o /boot/grub2/grub.cfg

That was enough for successful booting the OS after reboot. Enjoy !

Sunday, April 3, 2022

Traceback (most recent call last): File "/bin/yum-config-manager", line 205, in repo.cfg.options, repo.iteritems, repo.optionobj, AttributeError: 'RhnRepo' object has no attribute 'cfg'

I got this error after upgrading Oracle Linux 6.10 to 7.9. I tried to work with repositories - I was going to disable old of OL6 and enable some new of OL7. Any manipulations with repositories using yum-config-manager command, led to this strange error :

# yum-config-manager --enable '*'

...

Traceback (most recent call last):
 File "/bin/yum-config-manager", line 205, in <module>
   repo.cfg.options, repo.iteritems, repo.optionobj,
AttributeError: 'RhnRepo' object has no attribute 'cfg'

The solution was to disable the plugin rhn (set enabled=0) in the /etc/yum/pluginconf.d/rhnplugin.conf: and repeat failed yum-config-manager command after that. Below is the context of modified rhnplugin.conf file :

$ cat /etc/yum/pluginconf.d/rhnplugin.conf
[main]
enabled = 0
gpgcheck = 1
timeout = 120

# You can specify options per channel, e.g.:
#
#[rhel-i386-server-5]
#enabled = 1
#
#[some-unsigned-custom-channel]
#gpgcheck = 0


Good Luck !


Wednesday, March 30, 2022

ORA-01113: file xxx needs media recovery when issuing the 'alter pluggable database enable recovery' command

In DataGuard environment, when one of PDBs are created with STANDBYS=NONE parameter (other situations also are possible), the PDB's datafiles are not physically created on the standby site. Switchover or failover can potentially lead to non-functioning application working with PDB. To overcome this, one should duplicate PDB to standby site. It can be easily accomplished by restoring PDB from primary site and continuing the recovery. But the other days I encountered the issue trying to enable back the recovery of PDB :

> alter pluggable database enable recovery ;
alter pluggable database enable recovery
*
ERROR at line 1:
ORA-01113: file 65 needs media recovery
ORA-01110: data file 65: '...'

It turned out that the CDB was opened in READ ONLY mode before. So, the database behaved as expected - tried to open PDB before enabling recovery (sounds unusual). The solution was simple - restart physical standby in mount mode and repeat the enable recovery.  

Good Luck !

Monday, February 14, 2022

opatchauto and Error in GetCurrentDir(): 13

Recently, cloning GI oracle home, I encountered into the strange error :

---------------------------Patching Failed---------------------------------
Command execution failed during patching in home: /u01/app/19/grid_1914, host: aaa
Command failed:  /u01/app/19.13.0/grid/oui/bin/runInstaller -updateNodeList ORACLE_HOME=/u01/app/19.13.0/grid CRS="false" -local
Command failure output:                                                                     
Error in GetCurrentDir(): 13
Error in GetCurrentDir(): 13
Error in GetCurrentDir(): 13                    
Starting Oracle Universal Installer...
                                                                                              
Checking swap space: 0 MB available, 500 MB required.    Failed <<<<

Some requirement checks failed. You must fulfill these requirements before

continuing with the installation,

The log of opatchauto session said that the installer couldn't create the temporary file for checking prerequisites before running opatch :

2022-02-14 19:49:06,805 INFO  [107] com.oracle.glcm.patch.auto.db.integration.controller.action.DBCommonPatchAction - shell-init: error retrieving current directory: getcwd: cannot access parent directories: Permission denied
sh: /command_output_20159: cannot create

Looked strange, isn't it ? The installer tried to create the file in the root (/) directory.

The solution was unexpectable : changing current working directory to $ORACLE_HOME.

Good Luck !