#3960: Bug in vdt-rotate-gram-logs

#

Mon Sep 15 09:30:45 2008

timm@fnal.gov - Ticket created

Subject:	Bug in vdt-rotate-gram-logs
Date:	Mon, 15 Sep 2008 09:17:10 -0500 (CDT)
To:	vdt-support@OPENSCIENCEGRID.ORG
From:	Steven Timm <timm@fnal.gov>

Download (untitled) / with headers
text/plain 622b

There is a bug in the vdt-rotate-gram-logs script. There are a couple
places where it is using a normal shell glob. If there
are a very large number of files that match the criteria, 6000 or so,
say, it will fail and say "argument list too long".

This has been a problem with the script ever since it was written.

Steve Timm

--
------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
timm@fnal.gov http://home.fnal.gov/~timm/
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader.

#	Tue Sep 16 14:33:41 2008	roy - Given to adesmet

#	Tue Sep 16 14:33:51 2008	roy - Priority changed from (no value) to '3'

#	Tue Sep 16 14:33:51 2008	roy - Fix scheduled CUR added

#	Tue Sep 16 14:38:32 2008	roy - Correspondence added		[Reply]
	Download (untitled) / with headers text/plain 864b On Mon Sep 15 09:30:45 2008, timm@fnal.gov wrote: > There is a bug in the vdt-rotate-gram-logs script. There are a couple > places where it is using a normal shell glob. If there > are a very large number of files that match the criteria, 6000 or so, > say, it will fail and say "argument list too long". I took a quick look and didn't see any globs. Do you know what command line is failing? Is it perhaps the one that begins "tar cz --remove-files..."? That looks like the likely candidate to me, and if it is the problem I agree it should be fixed. I've assigned this to Alan De Smet, and future correspondence about this problemm will be with him. -alain ----------------------------------------------------------------- Alain Roy vdt-support@opensciencegrid.org VDT Support http://vdt.cs.wisc.edu/support.html

#	Tue Sep 16 14:38:33 2008	RT_System - Status changed from 'new' to 'open'

#	Tue Sep 16 15:26:18 2008	adesmet - Correspondence added		[Reply]
	Download (untitled) / with headers text/plain 206b I'm with Alain. Reviewing the script, only the tar command leaps out as a possible problem. The fix seems straightforward, but before I leap in I would like to confirm that it's the problem you're seeing.

#

Tue Sep 16 16:03:36 2008

timm@fnal.gov - Correspondence added

[Reply]

Subject:	Re: [vdt-support #3960] Bug in vdt-rotate-gram-logs
Date:	Tue, 16 Sep 2008 15:57:25 -0500 (CDT)
To:	Alan De Smet via RT <vdt-support@OPENSCIENCEGRID.ORG>
From:	Steven Timm <timm@fnal.gov>

Download (untitled) / with headers
text/plain 1k

Alan--I don't actually remember from the error message where the
command is failing. It was a while ago when I saw the error
message. the tar looks like the place but I can't say for sure.
I only discovered the bug because I forgot to enable a cron that
usually runs daily, for about 3 weeks and that was the only
thing that gave me so many files.

Steve

------------------------------------------------------------------
Steven C. Timm, Ph.D (630) 840-8525
timm@fnal.gov http://home.fnal.gov/~timm/
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader.

On Tue, 16 Sep 2008, Alan De Smet via RT wrote:

> I'm with Alain. Reviewing the script, only the tar command leaps out as
> a possible problem. The fix seems straightforward, but before I leap in
> I would like to confirm that it's the problem you're seeing.
>
> --
> View ticket at <http://crt.cs.wisc.edu/Ticket/Display.html?user=guest&pass=guest&id=3960>
> VDT Support: vdt-support@opensciencegrid.org
>

#

Wed Sep 24 16:47:30 2008

adesmet - Comments added

[Reply]

Subject:	[vdt-support #3960] SVN commit, rev 8080
To:	vdt-support@cs.wisc.edu
From:	adesmet@cs.wisc.edu

Download (untitled) / with headers
text/plain 503b

Commit comment:

I confirmed that a large number of gram_job_mgr files to be archived can cause
the arguments to tar to be too long and the command to fail. With this change,
the list of files is now written into a temporary file and that file passed in
via "-T" in tar. The new limit is based on the free disk space in /tmp, or
filesystem limits.

Changed files:
U vdt/branches/vdt-1.10.1/VDT-Logrotate/vdt/bin/vdt-rotate-gram-logs

To generate a diff:
svn diff -c 8080 file:///p/vdt/workspace/svn

#	Wed Sep 24 17:04:28 2008	adesmet - Correspondence added		[Reply]
	Download (untitled) / with headers text/plain 344b For the curious: ARG_MAX on Linux is usually 131072 (2^17, oddly enough). gram_job_mgr logs typically have lengths about 23 bytes long (gram_job_mgr_XXXXXX.log). 131072/23 is about 5,698, or about the 6,000 where Timm noticed the problem. The actual number is even smaller, taking into account the other arguments to tar and the environment.

#	Wed Sep 24 17:12:36 2008	adesmet - Comments added		[Reply]
	Download (untitled) / with headers text/plain 23b Implemented for 1.10.1k

#	Wed Sep 24 17:12:47 2008	adesmet - Status changed from 'open' to 'resolved'

#

Thu Sep 25 16:05:58 2008

adesmet - Comments added

[Reply]

Subject:	[vdt-support #3960] SVN commit, rev 8088
To:	vdt-support@cs.wisc.edu
From:	adesmet@cs.wisc.edu

Download (untitled) / with headers
text/plain 353b

Commit comment:

The TMPDIR option to File::Temp::tempfile is too new for us to rely on. Fortunately it's easy to work around; all it does is call into File::Spec::tmpdir and use it as DIR. Fixed.

Changed files:
U vdt/branches/vdt-1.10.1/VDT-Logrotate/vdt/bin/vdt-rotate-gram-logs

To generate a diff:
svn diff -c 8088 file:///p/vdt/workspace/svn