From zer0python at gmail.com Thu Sep 2 16:02:29 2010 From: zer0python at gmail.com (Joseph Kinsella) Date: Thu, 2 Sep 2010 16:02:29 -0400 Subject: [Lugstuff] geeknic Message-ID: This is kind of a late notice, but I figured I'd share with everyone that is in the Linux Users Group. There is something called a geeknic [ http://www.geeknic.org/ ] going on next Saturday (Sept 11, 2010) It is a bit of a drive (Oaks, PA) However, anyone who would like to come is welcome. There will be children there as well so it's kind of a family atmosphere. If anyone is interested in going feel free to email me or contact the Geeknic group. If you are interesting and going but do not want to drive or can not find a ride I can possibly help that situation, however I do drive a rather small car, so space is limited. You can view direction information here: http://wiki.geeknic.org/index.php/Lower_Perkiomen_Geeknic It is about a 2.5 hour drive from Annapolis, MD The geeknic team is working hard to ensure that there is enough food and fun for everyone, so if you intend to come it would be greatly appreciated if you let someone know that you are coming. I would also like to note that this is being done out of the pockets of many Open Source Fanatics, so if you would like to bring food, drinks, snacks, or whatever it would be appreciated as well but not a requirement. From tjk at annapolislinux.org Sun Sep 12 22:20:21 2010 From: tjk at annapolislinux.org (Theodore Knab) Date: Sun, 12 Sep 2010 22:20:21 -0400 Subject: [Lugstuff] next meeting Monday Sept 13 not Sept 14 Message-ID: <20100913022014.GA2127@annapolislinux.org> Hi, I due to the holiday, we have a meeting on 13 Sept. The website has 14 Sept on it, Monday is the 13 Sept. Please come on Monday Sept 13 2010 at 7PM to the LUG. Thanks and I apologize for the confusion. -- Ted Knab From tjk at annapolislinux.org Thu Sep 16 08:43:31 2010 From: tjk at annapolislinux.org (Theodore Knab) Date: Thu, 16 Sep 2010 08:43:31 -0400 Subject: [Lugstuff] shared memory kernel Message-ID: <20100916124331.GA10219@annapolislinux.org> Hi I was wondering if anyone knows how to tune the three shared memory parameters found in the /etc/sysctl.conf. For those of you not familiar with sysctl, it holds tunable kernel configurations. http://en.wikipedia.org/wiki/Sysctl Normally, you have to increase the default settings for shared memory use when messing with database applications and other applications that lock a lot of memory. The three I am focusing on are: kernel.shmax kernel.shmall kernel.shmni More specifically, I am wondering what happens if the tuning numbers for shmax are set too high ? Does the kernel think it has more RAM that it does ? The reverse seems to be true. For example, if the shared memory number is too low applications requiring more shared memory will crash or just not start. For example, my home system runs Zone Minder for keeping track of camera data. The default settings for shmax of 32Mb had to be increased for this application to work. In contrast, I found a Informix machine that was tuned with a shmmax that I thing was outrageously high. For example, I found a system with 24GB had this parameter: kernel.shmmax = 4398046511104 According to the Google calculation this number translates to: 4 398 046 511 104 bytes = 4096 gigabytes This is the default setting that IBM Informix has on their website for their Database application. So, I can understand how this may have been set. However, it seems not many people would have 4096GB or more on their machine. http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp?topic=/com.ibm.relnotes.doc/ids_1150xc5/mach/ids_machine_notes_11.50.linppc64.html I have since reset the setting to 75% of the total RAM in bytes, but I am still wondering what happens to a Linux system. The machine in question locked up twice due to all the memory being used. This is not something that created a log entry. Also, the machine seemed to become unstable when it started swapping. It was just as if all the applications on the machine were trying to consuming more RAM that the system had. For example, recently while running a database upgrade script on the machine, I could not run simple shell commands. When I tried to log in, I could type a few characters but only after a few minutes of waiting. I tried to type 'ctrl-alt-del' but it appeared the program the shutdown program could not be run in the time I had available to wait. It was not a hard crash where errors appear. Could this be a possible symptom of the kernel.shmmax being too high ? Here are the current setting I am using for the 24GB machine. kernel.shmmax = 18956485632 kernel.shmall = 4194304 kernel.shmmni = 4096 I used this script to calculate the max shared memory: #!/bin/sh #limits the kernel's shared memory to 1/2 actually physical memory mem_bytes=`awk '/MemTotal:/ { printf "%0.f",$2 * 1024}' /proc/meminfo` mem_max=`expr $mem_bytes / 2` #devide by 2 mem_max=`expr $mem_bytes / 4` #devide by 3 echo "mem max is $mem_max" mem_max=`expr $mem_max + $mem_max + $mem_max` #gets a 75% page_size=`getconf PAGE_SIZE` shmall=`expr $mem_bytes / $page_size` echo \# Maximum shared segment size in bytes echo kernel.shmmax = $mem_max echo \# Maximum number of shared memory segments in pages echo kernel.shmall = $shmall -- Ted Knab From tjk at annapolislinux.org Thu Sep 16 17:58:57 2010 From: tjk at annapolislinux.org (Theodore Knab) Date: Thu, 16 Sep 2010 17:58:57 -0400 Subject: [Lugstuff] shared memory kernel In-Reply-To: <2AFD48EC-F3CC-4AA6-9FC7-FEDECACECF4E@gmail.com> References: <20100916124331.GA10219@annapolislinux.org> <2AFD48EC-F3CC-4AA6-9FC7-FEDECACECF4E@gmail.com> Message-ID: <20100916215856.GA11783@annapolislinux.org> Thanks Thomas, I saw that article before, but I re-read it again. It seems paging hell seems to be the result of setting too much shared memory. But, what is paging hell ? On 16/09/10 09:31 -0400, Thomas Gallen wrote: > I believe this will answer most of your questions: > > http://www.pythian.com/news/245/the-mysterious-world-of-shmmax-and-shmall/ > > Thomas > > On Sep 16, 2010, at 8:43 AM, Theodore Knab wrote: > > > Hi I was wondering if anyone knows how to tune the three shared memory parameters > > found in the /etc/sysctl.conf. For those of you not familiar with > > sysctl, it holds tunable kernel configurations. > > > > http://en.wikipedia.org/wiki/Sysctl > > > > Normally, you have to increase the default settings for shared > > memory use when messing with database applications and other applications that lock a lot > > of memory. > > > > The three I am focusing on are: > > kernel.shmax > > kernel.shmall > > kernel.shmni > > > > More specifically, I am wondering what happens if the tuning numbers for > > shmax are set too high ? Does the kernel think it has more RAM that it > > does ? The reverse seems to be true. For example, if the shared memory > > number is too low applications requiring more shared memory will crash > > or just not start. > > > > For example, my home system runs Zone Minder for keeping track of > > camera data. The default settings for shmax of 32Mb had to be > > increased for this application to work. > > > > In contrast, I found a Informix machine that was tuned with a > > shmmax that I thing was outrageously high. > > > > For example, I found a system with 24GB had this parameter: > > kernel.shmmax = 4398046511104 > > > > According to the Google calculation this number translates to: > > 4 398 046 511 104 bytes = 4096 gigabytes > > > > This is the default setting that IBM Informix has on their website for > > their Database application. So, I can understand how this may have been > > set. However, it seems not many people would have 4096GB or more on their machine. > > > > http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp?topic=/com.ibm.relnotes.doc/ids_1150xc5/mach/ids_machine_notes_11.50.linppc64.html > > > > I have since reset the setting to 75% of the total RAM in bytes, but I am still > > wondering what happens to a Linux system. The machine in question locked > > up twice due to all the memory being used. This is not something that > > created a log entry. Also, the machine seemed to become unstable when it started swapping. > > It was just as if all the applications on the machine were trying to consuming more RAM > > that the system had. For example, recently while running a database upgrade script on the > > machine, I could not run simple shell commands. When I tried to log in, > > I could type a few characters but only after a few minutes of waiting. > > I tried to type 'ctrl-alt-del' but it appeared the program the shutdown > > program could not be run in the time I had available to wait. > > It was not a hard crash where errors appear. > > > > Could this be a possible symptom of the kernel.shmmax being too high ? > > > > Here are the current setting I am using for the 24GB machine. > > kernel.shmmax = 18956485632 > > kernel.shmall = 4194304 > > kernel.shmmni = 4096 > > > > I used this script to calculate the max shared memory: > > #!/bin/sh > > #limits the kernel's shared memory to 1/2 actually physical memory > > mem_bytes=`awk '/MemTotal:/ { printf "%0.f",$2 * 1024}' /proc/meminfo` > > mem_max=`expr $mem_bytes / 2` #devide by 2 > > mem_max=`expr $mem_bytes / 4` #devide by 3 > > > > echo "mem max is $mem_max" > > > > mem_max=`expr $mem_max + $mem_max + $mem_max` #gets a 75% > > page_size=`getconf PAGE_SIZE` > > shmall=`expr $mem_bytes / $page_size` > > echo \# Maximum shared segment size in bytes > > echo kernel.shmmax = $mem_max > > echo \# Maximum number of shared memory segments in pages > > echo kernel.shmall = $shmall > > > > > > -- > > Ted Knab > > > > _______________________________________________ > > Lugstuff mailing list > > Lugstuff at annapolislinux.org > > http://list.annapolislinux.org/cgi-bin/mailman/listinfo/lugstuff > -- Ted Knab Stevensville, MD USA From tjk at annapolislinux.org Thu Sep 16 18:01:32 2010 From: tjk at annapolislinux.org (Theodore Knab) Date: Thu, 16 Sep 2010 18:01:32 -0400 Subject: [Lugstuff] shared memory kernel In-Reply-To: <47872.63.238.140.254.1284664604.squirrel@etome.net> References: <20100916124331.GA10219@annapolislinux.org> <47872.63.238.140.254.1284664604.squirrel@etome.net> Message-ID: <20100916220131.GB11783@annapolislinux.org> Thanks Nick, Actually, Oracle recommends the kernel.shmmax be set to half the usable RAM in bytes. This is completely different than what the Informix documentation people put out and make more sense. On 16/09/10 12:16 -0700, Nicholas S. Rosasco wrote: > Not sure... > you might try hunting around the Oracle forums, as ORA's DB requires these > to be seriously cranked prior to install. > > Shrug, curious to know m'self, > Nick > > > On Thu, September 16, 2010 5:43 am, Theodore Knab wrote: > > Hi I was wondering if anyone knows how to tune the three shared memory > > parameters found in the /etc/sysctl.conf. For those of you not familiar > > with sysctl, it holds tunable kernel configurations. > > > > http://en.wikipedia.org/wiki/Sysctl > > > > > > Normally, you have to increase the default settings for shared > > memory use when messing with database applications and other applications > > that lock a lot of memory. > > > > The three I am focusing on are: > > kernel.shmax kernel.shmall kernel.shmni > > > > More specifically, I am wondering what happens if the tuning numbers for > > shmax are set too high ? Does the kernel think it has more RAM that it does > > ? The reverse seems to be true. For example, if the shared memory > > number is too low applications requiring more shared memory will crash or > > just not start. > > > > For example, my home system runs Zone Minder for keeping track of > > camera data. The default settings for shmax of 32Mb had to be increased for > > this application to work. > > > > In contrast, I found a Informix machine that was tuned with a > > shmmax that I thing was outrageously high. > > > > For example, I found a system with 24GB had this parameter: > > kernel.shmmax = 4398046511104 > > > > According to the Google calculation this number translates to: > > 4 398 046 511 104 bytes = 4096 gigabytes > > > > > > This is the default setting that IBM Informix has on their website for > > their Database application. So, I can understand how this may have been > > set. However, it seems not many people would have 4096GB or more on their > > machine. > > > > http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp?topic=/co > > m.ibm.relnotes.doc/ids_1150xc5/mach/ids_machine_notes_11.50.linppc64.html > > > > > > I have since reset the setting to 75% of the total RAM in bytes, but I am > > still wondering what happens to a Linux system. The machine in question > > locked up twice due to all the memory being used. This is not something > > that created a log entry. Also, the machine seemed to become unstable when > > it started swapping. It was just as if all the applications on the machine > > were trying to consuming more RAM that the system had. For example, > > recently while running a database upgrade script on the machine, I could > > not run simple shell commands. When I tried to log in, I could type a few > > characters but only after a few minutes of waiting. I tried to type > > 'ctrl-alt-del' but it appeared the program the shutdown > > program could not be run in the time I had available to wait. It was not a > > hard crash where errors appear. > > > > Could this be a possible symptom of the kernel.shmmax being too high ? > > > > > > Here are the current setting I am using for the 24GB machine. > > kernel.shmmax = 18956485632 kernel.shmall = 4194304 kernel.shmmni = 4096 > > > > I used this script to calculate the max shared memory: > > #!/bin/sh > > #limits the kernel's shared memory to 1/2 actually physical memory > > mem_bytes=`awk '/MemTotal:/ { printf "%0.f",$2 * 1024}' /proc/meminfo` > > mem_max=`expr $mem_bytes / 2` #devide by 2 mem_max=`expr $mem_bytes / 4` > > #devide by 3 > > > > > > echo "mem max is $mem_max" > > > > mem_max=`expr $mem_max + $mem_max + $mem_max` #gets a 75% > > page_size=`getconf PAGE_SIZE` shmall=`expr $mem_bytes / $page_size` echo \# > > Maximum shared segment size in bytes > > echo kernel.shmmax = $mem_max echo \# Maximum number of shared memory > > segments in pages echo kernel.shmall = $shmall > > > > > > -- > > Ted Knab > > > > > > _______________________________________________ > > Lugstuff mailing list > > Lugstuff at annapolislinux.org > > http://list.annapolislinux.org/cgi-bin/mailman/listinfo/lugstuff > > > > > > > -- Ted Knab Stevensville, MD USA From kaori.hinata at gmail.com Thu Sep 16 20:10:27 2010 From: kaori.hinata at gmail.com (Thomas Gallen) Date: Thu, 16 Sep 2010 20:10:27 -0400 Subject: [Lugstuff] shared memory kernel In-Reply-To: <20100916215856.GA11783@annapolislinux.org> References: <20100916124331.GA10219@annapolislinux.org> <2AFD48EC-F3CC-4AA6-9FC7-FEDECACECF4E@gmail.com> <20100916215856.GA11783@annapolislinux.org> Message-ID: <20100917001025.GA2866@localhost.lan.merumeru.mobi> Paging hell refers to (I believe) the kernel swapping out pages and swapping in pages (page in/page out) rapidly due to programs allocating too much memory (again, I believe). The translation between virtual memory addresses and physical memory addresses allows Linux to give programs the illusion that there's much more memory available than there actually is. The intent is to allow the kernel to manage the allocation, deallocation and swapping of chunks of continuous pages transparently so that the user doesn't need to know that the seemingly linear memory chunk they have allocated may actually be spread out between multiple pages that are not necessarily linear in physical memory. The additional more obviously consequence of swapping pages into and out of swap is that, as it's bound to a physical hard disk, the performance impact is rather horrific. On another note, unless I'm mistaken as to the meaning of the statistics, vmstat will allow you to monitor the page in and page out (though I think it calls them swap in (si) and swap out (so)) rates as currently reported by the kernel so if you want to try playing with the sysctl values while doing some load tests and keeping a close eye on vmstat then perhaps that may help you find the sweet spot you're looking for. Disclaimer: I'm not a Linux kernel developer am still trying to wrap my head around all of the Linux memory management stuff myself. Some interesting reading: http://scriptmatrix.net/cs2106/wiki/index.php?title=Buddy_Allocator http://linux-mm.org/PageAllocation Thomas On Thu, Sep 16, 2010 at 05:58:57PM -0400, Theodore Knab wrote: > Thanks Thomas, > > I saw that article before, but I re-read it again. > > It seems paging hell seems to be the result of setting too much shared > memory. > > But, what is paging hell ? > > > On 16/09/10 09:31 -0400, Thomas Gallen wrote: > > I believe this will answer most of your questions: > > > > http://www.pythian.com/news/245/the-mysterious-world-of-shmmax-and-shmall/ > > > > Thomas > > > > On Sep 16, 2010, at 8:43 AM, Theodore Knab wrote: > > > > > Hi I was wondering if anyone knows how to tune the three shared memory parameters > > > found in the /etc/sysctl.conf. For those of you not familiar with > > > sysctl, it holds tunable kernel configurations. > > > > > > http://en.wikipedia.org/wiki/Sysctl > > > > > > Normally, you have to increase the default settings for shared > > > memory use when messing with database applications and other applications that lock a lot > > > of memory. > > > > > > The three I am focusing on are: > > > kernel.shmax > > > kernel.shmall > > > kernel.shmni > > > > > > More specifically, I am wondering what happens if the tuning numbers for > > > shmax are set too high ? Does the kernel think it has more RAM that it > > > does ? The reverse seems to be true. For example, if the shared memory > > > number is too low applications requiring more shared memory will crash > > > or just not start. > > > > > > For example, my home system runs Zone Minder for keeping track of > > > camera data. The default settings for shmax of 32Mb had to be > > > increased for this application to work. > > > > > > In contrast, I found a Informix machine that was tuned with a > > > shmmax that I thing was outrageously high. > > > > > > For example, I found a system with 24GB had this parameter: > > > kernel.shmmax = 4398046511104 > > > > > > According to the Google calculation this number translates to: > > > 4 398 046 511 104 bytes = 4096 gigabytes > > > > > > This is the default setting that IBM Informix has on their website for > > > their Database application. So, I can understand how this may have been > > > set. However, it seems not many people would have 4096GB or more on their machine. > > > > > > http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp?topic=/com.ibm.relnotes.doc/ids_1150xc5/mach/ids_machine_notes_11.50.linppc64.html > > > > > > I have since reset the setting to 75% of the total RAM in bytes, but I am still > > > wondering what happens to a Linux system. The machine in question locked > > > up twice due to all the memory being used. This is not something that > > > created a log entry. Also, the machine seemed to become unstable when it started swapping. > > > It was just as if all the applications on the machine were trying to consuming more RAM > > > that the system had. For example, recently while running a database upgrade script on the > > > machine, I could not run simple shell commands. When I tried to log in, > > > I could type a few characters but only after a few minutes of waiting. > > > I tried to type 'ctrl-alt-del' but it appeared the program the shutdown > > > program could not be run in the time I had available to wait. > > > It was not a hard crash where errors appear. > > > > > > Could this be a possible symptom of the kernel.shmmax being too high ? > > > > > > Here are the current setting I am using for the 24GB machine. > > > kernel.shmmax = 18956485632 > > > kernel.shmall = 4194304 > > > kernel.shmmni = 4096 > > > > > > I used this script to calculate the max shared memory: > > > #!/bin/sh > > > #limits the kernel's shared memory to 1/2 actually physical memory > > > mem_bytes=`awk '/MemTotal:/ { printf "%0.f",$2 * 1024}' /proc/meminfo` > > > mem_max=`expr $mem_bytes / 2` #devide by 2 > > > mem_max=`expr $mem_bytes / 4` #devide by 3 > > > > > > echo "mem max is $mem_max" > > > > > > mem_max=`expr $mem_max + $mem_max + $mem_max` #gets a 75% > > > page_size=`getconf PAGE_SIZE` > > > shmall=`expr $mem_bytes / $page_size` > > > echo \# Maximum shared segment size in bytes > > > echo kernel.shmmax = $mem_max > > > echo \# Maximum number of shared memory segments in pages > > > echo kernel.shmall = $shmall > > > > > > > > > -- > > > Ted Knab > > > > > > _______________________________________________ > > > Lugstuff mailing list > > > Lugstuff at annapolislinux.org > > > http://list.annapolislinux.org/cgi-bin/mailman/listinfo/lugstuff > > > > -- > Ted Knab > Stevensville, MD USA > > _______________________________________________ > Lugstuff mailing list > Lugstuff at annapolislinux.org > http://list.annapolislinux.org/cgi-bin/mailman/listinfo/lugstuff From tjk at annapolislinux.org Wed Sep 22 14:11:10 2010 From: tjk at annapolislinux.org (Theodore Knab) Date: Wed, 22 Sep 2010 14:11:10 -0400 Subject: [Lugstuff] special events: speakers Message-ID: <20100922181109.GB27095@annapolislinux.org> I have some good news. We have SUSE coming to talk at the LUG on Oct 4th. Additionally, Jared Smith, Fedora project leader, on Monday Nov 1st. Both should be interesting. The details can be found here: Oct 4 2010 SUSE: http://annapolislinux.org/?p=292 Nov 1 2010 Fedora: http://annapolislinux.org/?p=296 -- Ted Knab Stevensville, MD USA From tjk at annapolislinux.org Wed Sep 22 14:41:39 2010 From: tjk at annapolislinux.org (Theodore Knab) Date: Wed, 22 Sep 2010 14:41:39 -0400 Subject: [Lugstuff] shared memory kernel In-Reply-To: <20100917001025.GA2866@localhost.lan.merumeru.mobi> References: <20100916124331.GA10219@annapolislinux.org> <2AFD48EC-F3CC-4AA6-9FC7-FEDECACECF4E@gmail.com> <20100916215856.GA11783@annapolislinux.org> <20100917001025.GA2866@localhost.lan.merumeru.mobi> Message-ID: <20100922184139.GA27181@annapolislinux.org> Thanks for this information. I can't say I really understand the Buddy Allocator. But, I understand more than I did last week which is an improvement. Do you know if there any tools that can be used to monitor memory fragmentation ? Also, is it better to give all the 'resident' RAM to the 'shared' pool? More specifically, in Linux it seems there is a kernel.shmmax, which is the max shared memory segment size any one process can lock. Also, there is the kernel.shmall which is the max. shared memory used on the system. If I am understanding the concepts right, giving the kernel the maximum shmall size will allow the system to run more efficiently. This will allow all the programs running to share the same memory pool as needed. Making the systems more happy. Currently, for a 24GB database machine running, apache, and sendmail, I changed the kernel.shmall value from a 16GB limit to a 24GB limit. Will this create issues ? Is there any time the OS needs to use 'resident' RAM over 'shared' RAM ? #source Informix defaults #max segment size is shmmax kernel.shmmax = 4398046511104 #max shared memory to use. #16GB RAM = 17179869184 bytes #We get this number by dividing the RAM in bytes by the system page size #page size is 4096 on most 64bit systems kernel.shmall = 4194304 #changed to kernel.shmmax = 4398046511104 #24GB to use all 24GB kernel.shmall = 6291456 On 16/09/10 20:10 -0400, Thomas Gallen wrote: > Paging hell refers to (I believe) the kernel swapping out pages and > swapping in pages (page in/page out) rapidly due to programs allocating > too much memory (again, I believe). The translation between virtual > memory addresses and physical memory addresses allows Linux to give > programs the illusion that there's much more memory available than > there actually is. The intent is to allow the kernel to manage the > allocation, deallocation and swapping of chunks of continuous pages > transparently so that the user doesn't need to know that the seemingly > linear memory chunk they have allocated may actually be spread out > between multiple pages that are not necessarily linear in physical > memory. The additional more obviously consequence of swapping pages > into and out of swap is that, as it's bound to a physical hard disk, > the performance impact is rather horrific. > > On another note, unless I'm mistaken as to the meaning of the > statistics, vmstat will allow you to monitor the page in and page out > (though I think it calls them swap in (si) and swap out (so)) rates as > currently reported by the kernel so if you want to try playing with the > sysctl values while doing some load tests and keeping a close eye on > vmstat then perhaps that may help you find the sweet spot you're > looking for. > > Disclaimer: I'm not a Linux kernel developer am still trying to wrap my > head around all of the Linux memory management stuff myself. Some > interesting reading: > > http://scriptmatrix.net/cs2106/wiki/index.php?title=Buddy_Allocator > http://linux-mm.org/PageAllocation > > Thomas > > On Thu, Sep 16, 2010 at 05:58:57PM -0400, Theodore Knab wrote: > > Thanks Thomas, > > > > I saw that article before, but I re-read it again. > > > > It seems paging hell seems to be the result of setting too much shared > > memory. > > > > But, what is paging hell ? > > > > > > On 16/09/10 09:31 -0400, Thomas Gallen wrote: > > > I believe this will answer most of your questions: > > > > > > http://www.pythian.com/news/245/the-mysterious-world-of-shmmax-and-shmall/ > > > > > > Thomas > > > > > > On Sep 16, 2010, at 8:43 AM, Theodore Knab wrote: > > > > > > > Hi I was wondering if anyone knows how to tune the three shared memory parameters > > > > found in the /etc/sysctl.conf. For those of you not familiar with > > > > sysctl, it holds tunable kernel configurations. > > > > > > > > http://en.wikipedia.org/wiki/Sysctl > > > > > > > > Normally, you have to increase the default settings for shared > > > > memory use when messing with database applications and other applications that lock a lot > > > > of memory. > > > > > > > > The three I am focusing on are: > > > > kernel.shmax > > > > kernel.shmall > > > > kernel.shmni > > > > > > > > More specifically, I am wondering what happens if the tuning numbers for > > > > shmax are set too high ? Does the kernel think it has more RAM that it > > > > does ? The reverse seems to be true. For example, if the shared memory > > > > number is too low applications requiring more shared memory will crash > > > > or just not start. > > > > > > > > For example, my home system runs Zone Minder for keeping track of > > > > camera data. The default settings for shmax of 32Mb had to be > > > > increased for this application to work. > > > > > > > > In contrast, I found a Informix machine that was tuned with a > > > > shmmax that I thing was outrageously high. > > > > > > > > For example, I found a system with 24GB had this parameter: > > > > kernel.shmmax = 4398046511104 > > > > > > > > According to the Google calculation this number translates to: > > > > 4 398 046 511 104 bytes = 4096 gigabytes > > > > > > > > This is the default setting that IBM Informix has on their website for > > > > their Database application. So, I can understand how this may have been > > > > set. However, it seems not many people would have 4096GB or more on their machine. > > > > > > > > http://publib.boulder.ibm.com/infocenter/idshelp/v115/index.jsp?topic=/com.ibm.relnotes.doc/ids_1150xc5/mach/ids_machine_notes_11.50.linppc64.html > > > > > > > > I have since reset the setting to 75% of the total RAM in bytes, but I am still > > > > wondering what happens to a Linux system. The machine in question locked > > > > up twice due to all the memory being used. This is not something that > > > > created a log entry. Also, the machine seemed to become unstable when it started swapping. > > > > It was just as if all the applications on the machine were trying to consuming more RAM > > > > that the system had. For example, recently while running a database upgrade script on the > > > > machine, I could not run simple shell commands. When I tried to log in, > > > > I could type a few characters but only after a few minutes of waiting. > > > > I tried to type 'ctrl-alt-del' but it appeared the program the shutdown > > > > program could not be run in the time I had available to wait. > > > > It was not a hard crash where errors appear. > > > > > > > > Could this be a possible symptom of the kernel.shmmax being too high ? > > > > > > > > Here are the current setting I am using for the 24GB machine. > > > > kernel.shmmax = 18956485632 > > > > kernel.shmall = 4194304 > > > > kernel.shmmni = 4096 > > > > > > > > I used this script to calculate the max shared memory: > > > > #!/bin/sh > > > > #limits the kernel's shared memory to 1/2 actually physical memory > > > > mem_bytes=`awk '/MemTotal:/ { printf "%0.f",$2 * 1024}' /proc/meminfo` > > > > mem_max=`expr $mem_bytes / 2` #devide by 2 > > > > mem_max=`expr $mem_bytes / 4` #devide by 3 > > > > > > > > echo "mem max is $mem_max" > > > > > > > > mem_max=`expr $mem_max + $mem_max + $mem_max` #gets a 75% > > > > page_size=`getconf PAGE_SIZE` > > > > shmall=`expr $mem_bytes / $page_size` > > > > echo \# Maximum shared segment size in bytes > > > > echo kernel.shmmax = $mem_max > > > > echo \# Maximum number of shared memory segments in pages > > > > echo kernel.shmall = $shmall > > > > > > > > > > > > -- > > > > Ted Knab > > > > > > > > _______________________________________________ > > > > Lugstuff mailing list > > > > Lugstuff at annapolislinux.org > > > > http://list.annapolislinux.org/cgi-bin/mailman/listinfo/lugstuff > > > > > > > -- > > Ted Knab > > Stevensville, MD USA > > > > _______________________________________________ > > Lugstuff mailing list > > Lugstuff at annapolislinux.org > > http://list.annapolislinux.org/cgi-bin/mailman/listinfo/lugstuff -- Ted Knab Stevensville, MD USA From tjk at annapolislinux.org Wed Sep 22 16:15:38 2010 From: tjk at annapolislinux.org (Theodore Knab) Date: Wed, 22 Sep 2010 16:15:38 -0400 Subject: [Lugstuff] speaker attendance registration Message-ID: <20100922201538.GA27764@annapolislinux.org> Hi, Since seats are limited please RSVP for the planned speaking events. Here is the link to RSVP the SUSE presenation: http://www.plugmyevent.com/events/view/lfz1C6rjyL Here is the link to RSVP the Fedora presenation: http://www.plugmyevent.com/events/view/vI8JNcEqZV -- Ted Knab