Recently i faced the below error while starting the test databases on Sun Solaris 10,
SQL> startup ORA-27102: out of memory Solaris-AMD64 Error: 22: Invalid argument
On asking to the team, came to know,that there was some issue, which they were not aware of.So it was time to investigate 🙂
Check the Alert log
Starting ORACLE instance (normal) Sun Jun 20 17:00:06 2010 WARNING: EINVAL creating segment of size 0x0000000180004000 fix shm parameters in /etc/system or equivalent
Oracle is trying to create a 6Gb shared memory segment, but operating system responded with an invalid argument (EINVAL) error message. As the alert log suggested, “fix shm parameters in /etc/system” and showed “creating segment of size 0x0000000180004000”.Converting the hex value (“180004000” ) in decimal comes up to “6442467328”.
As suggested, changed the value of set shmsys:shminfo_shmmax to “6442467400” in /etc/system and rebooted the server.Again,the same error
-bash-3.00$ sqlplus / as sysdba SQL*Plus: Release 10.2.0.4.0 - Production on Sun Jun 20 17:15:03 2010 Copyright (c) 1982, 2007, Oracle. All Rights Reserved. Connected to an idle instance. SQL> startup ORA-27102: out of memory Solaris-AMD64 Error: 22: Invalid argument SQL> exit Disconnected -bash-3.00$
Immediately, i remembered in Sun Solaris 10, they have created the resource control utility, which can be checked using prctl command
http://docs.sun.com/app/docs/doc/817-1592/rmctrls.task-33?a=view
-bash-3.00$ cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3:::: group.staff:10:::: oracle:100::::project.max-shm-memory=(priv,4294967296,deny) -bash-3.00$
So the system is using the default value of 4Gb in this scenario, where as Oracle is trying to create a memory segment of almost 6Gb, larger than 4Gb. After few searches on google, got the command to change the value.Logged in as root (luckily had the password :)), and used the below command
bash-3.00# projmod -s -K “project.max-shm-memory=(priv,6442467400,deny)” oracle
Then, tried to startup the database,but again failure.Checking back the parameters
-bash-3.00$ cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3:::: group.staff:10:::: oracle:100::::project.max-shm-memory=(priv,6442467400,deny)
-bash-3.00$ prctl -i project oracle
project: 100: oracle
NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT
project.max-contracts
privileged 10.0K - deny -
system 2.15G max deny -
project.max-device-locked-memory
privileged 1002MB - deny -
system 16.0EB max deny -
project.max-locked-memory
system 16.0EB max deny -
project.max-port-ids
privileged 8.19K - deny -
system 65.5K max deny -
project.max-shm-memory
privileged 6.00GB - deny -
system 16.0EB max deny -
project.max-shm-ids
...............
The above showed,oracle is privileged 6.00Gb for max-shm-memory,but the hex value in alert log is little more than 6.00Gb, so allocated oracle 6.5Gb and tried
bash-3.00# projmod -s -K “project.max-shm-memory=(priv,6.5GB,deny)” oracle
bash-3.00# cat /etc/project system:0:::: user.root:1:::: noproject:2:::: default:3:::: group.staff:10:::: oracle:100::::project.max-shm-memory=(priv,6979321856,deny) bash-3.00#
Database startup
-bash-3.00$ sqlplus / as sysdba SQL*Plus: Release 10.2.0.4.0 - Production on Sun Jun 20 17:30:33 2010 Copyright (c) 1982, 2007, Oracle. All Rights Reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 6442450944 bytes Fixed Size 2093584 bytes Variable Size 889196016 bytes Database Buffers 5536481280 bytes Redo Buffers 14680064 bytes Database mounted. Database opened. SQL> SQL>
Learn. Share. Repeat. 
Thanks, it works for me.
In my environment, I change max-shm-memory from 4gb to 4.5gb and solve same issue:
prctl -n project.max-shm-memory -v 4.5gb -r -i project user.oracle
But, the alter log indicated while out-of-memory error happened:
WARNING: EINVAL creating segment of size 0x0000000069000000
fix shm parameters in /etc/system or equivalent
If I follow your post, I should not change it because I have enough max-shm-memory.