Skip Menu | Logged in as guest | Logout
 
Ticket metadata
Id: 3671
Status: resolved
Priority: 3/0
Queue: vdt-support

Fixed in: (no value)
Fix scheduled: CUR

Owner: Alan De Smet
Requestors: Alan.Sill@ttu.edu
ASim@lbl.gov
Cc: smartin@mcs.anl.gov
AdminCc:

More about Alan.Sill@ttu.edu
Comments about this user:
No comment entered about this user
This user's 10 highest priority tickets:
Groups this user belongs to:
  • Everyone

More about ASim@lbl.gov
Comments about this user:
No comment entered about this user
This user's 10 highest priority tickets:
Groups this user belongs to:
  • Everyone

New reminder:

Created: Mon Jul 14 13:40:47 2008
Starts: Not set
Started: Not set
Last Contact: Mon Sep 22 18:15:44 2008
Due: Not set
Closed: Mon Sep 29 09:59:41 2008
Updated: Mon Sep 29 09:59:43 2008 by adesmet



History Brief headersFull headers
CC: Alan Sill <Alan.Sill@ttu.edu>, "SRM Support @ LBNL" <srm@lbl.gov>, vdt-support <vdt-support@OPENSCIENCEGRID.ORG>
Subject: Re: sigmorgh gsiftp
Date: Mon, 14 Jul 2008 13:38:55 -0500
To: asim@lbl.gov
From: Alan Sill <Alan.Sill@ttu.edu>
Note to VDT team: we seem to be experinecing problems with a BeStMan
installation out of VDT 1.10.1d that points to problems in the gridftp
libraries, see below. Running vdt-verson reports the following
installed set:

You have installed a subset of VDT version 1.10.1d:
Berkeley Storage Manager (BeStMan) 2.2.0.10
CA Certificates v37 (includes IGTF 1.22 CAs)
Fetch CRL 2.6.6
Globus Toolkit, web-services, client 4.0.7
Globus Toolkit, web-services, server 4.0.7
GPT 3.2autotools2004-NMI-9.0
GSI-Enabled OpenSSH 4.3
Grid User Management System (GUMS) Client 1.2.16
Java 5 SDK 1.5.0_14
Logrotate 3.7
MySQL 4.1.22
MySQL Connector/J 5.0.6
PRIMA Authorization Module 0.7.1
PRIMA Authorization Module For GT4 Web Services 0.1.2

Alex: notes problems doing remote gridftp transfers, see below. I
can log in to the machine directly and see the directories in question
from a user account, but I see problems even using a simple gridftp
client like uberftp in accessing any directory:

uberftp> ls
Could not list ls: 500-Command failed. :
globus_gridftp_server_file.c:globus_l_gfs_file_stat:389:
500-System error in stat: No such file or directory
500-A system call failed: No such file or directory
500 End.

Looking around the web, I see that this message has been associated
with GT 4.0.7, e.g. see the following message from Stuart Martin
(truncated) to the gt-user list in early June:

> Try install the 4.0.7 RFT update package
> (globus_wsrf_rft_service_java-0.39 ) from http://www.globus.org/toolkit/advisories.html
> Looks like that could be the problem.
>
> -Stu
>
> On Jun 5, 2008, at Jun 5, 4:23 PM, Patrick Armstrong wrote:
>
>> Hi.
>>
>> I've just installed Globus 4.0.7 from source on two Scientific
>> Linux 5.1
>> machines, and am having a strange problem when staging files back
>> from
>> the execution machine. heplw26 is the workstation I am submitting the
>> job from, and gridhn is the execution host. My certificate is
>> mapped to
>> dev08 on gridhn and patricka on heplw26.
>>
>> When I run the following job, I get an error staging back:
>> [...]
>> This is what happens when I run the job:
>>
>> [patricka@heplw26 Desktop]$ globusrun-ws -submit -s -F
>> gridhn.phys.uvic.ca -Ft Fork -f testjob.xml
>> Delegating user credentials...Done.
>> Submitting job...Done.
>> Job ID: uuid:8295dc48-3342-11dd-bbae-001e8c04a58b
>> Termination time: 06/06/2008 21:01 GMT
>> Current job state: StageIn
>> Current job state: StageOut
>> dev08
>> Current job state: Failed
>> Destroying job...Done.
>> Cleaning up any delegated credentials...Done.
>> globusrun-ws: Job failed: Staging error for RSL element fileStageOut.
>> Can't do MLST on non-existing file/dir /home/dev08/testout.txt on
>> server
>> heplw26.phys.UVic.CA [Caused by: Server refused performing the
>> request.
>> Custom message: Server refused MLST command (error code 1) [Nested
>> exception message: Custom message: Unexpected reply: 500-Command
>> failed : System error in stat: No such file or directory
>> 500-A system call failed: No such file or directory
>> 500 End.]]
>>
>>
>> The strange thing is, /home/dev08/testout.txt *does* exist on gridhn

Question to the VDT team: have other people reported this problem?

I verified that I get the same problem on our ITB and production OSG
1.0 systems also, but only from my Mac OS X version of the VDT
client. Updating this client to 1.10.1e does not help; I still get
the same problem. I also verified that I do NOT get the same problem
when using uberftp from, e.g., my OSG ITB system to my OSG production
system. So there seems to be some kind of inconsistency.

Is it possible that the 4.0.7 RFT update package
(globus_wsrf_rft_service_java-0.39 ) from http://www.globus.org/toolkit/advisories.html
is not uniformly distributed and applied, e.g. in the SRM client and
VDT client packages?

Thanks,
Alan

On Jul 14, 2008, at 12:35 PM, Alex Sim wrote:

> Hi Alan,
>
> it's still hanging at STOR even if it's in PASV mode.... maybe it's
> GLOBUS_TCP_SOURCE_RANGE to test?
> Is my path on sigmorgh okay (/mnt/hep/osg/asim.test.data)?
>
> thanks
>
>
> debug: sending command:
> TYPE I
> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
> :
> 200 Type set to I.
>
> debug: sending command:
> PBSZ 16384
>
> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
> :
> 200 PBSZ=16384
>
> debug: sending command:
> PASV
>
> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
> :
> 227 Entering Passive Mode (129,118,104,41,113,234)
>
> debug: sending command:
> STOR //mnt/hep/osg/asim.test.data
>
>
>
> -- Alex
> asim at lbl dot gov
>
>
>
> On 7/13/08 9:16 PM, Alan Sill wrote:
>> Hi Alex,
>>
>> I forgot to set the globus tcp port range in the $VDT_LOCATION/vdt/
>> etc/vdt-local-setup.(sh,csh) files on sigmorgh after the last
>> upgrade. This messes up some of globus's call-back functions.
>>
>> Try it now and let me know if it works.
>>
>> Thanks,
>> Alan
>>
>> On Jul 11, 2008, at 3:58 PM, Alex Sim wrote:
>>
>>> do you have any idea on my gridftp hanging to sigmorgh?
>>> I was trying to put a file into /mnt/hep/osg/asim.test.data.
>>> it seems authN was okay... and at the STOR, it was hanging....
>>> thanks
>>
>> Alan Sill, Ph.D
>> TIGRE Senior Scientist, High Performance Computing Center
>> Adjunct Professor of Physics
>> TTU
>>
>> ====================================================================
>> : Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
>> : e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
>> ====================================================================
>>
>>
>>

Alan Sill, Ph.D
TIGRE Senior Scientist, High Performance Computing Center
Adjunct Professor of Physics
TTU

====================================================================
: Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
: e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
====================================================================
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Tue, 15 Jul 2008 11:10:09 +0200
To: vdt-support@OPENSCIENCEGRID.ORG
From: Alain Roy <roy@cs.wisc.edu>
Download (untitled) / with headers
text/plain 6.6k
Hi Alan,

Thanks for reporting the problem. I've added Stu Martin to this ticket.

Stu--is Alan Sill's analysis (below) correct? I thought the VDT had
the important bits of the RFT patch:

http://vdt.cs.wisc.edu/patches/1.10.1/

Are we missing something that we should have?

Thanks,
-alain

-----------------------------------------------------------------
Alain Roy vdt-support@opensciencegrid.org
VDT Support http://vdt.cs.wisc.edu/support.html

> Note to VDT team: we seem to be experinecing problems with a BeStMan
> installation out of VDT 1.10.1d that points to problems in the gridftp
> libraries, see below. Running vdt-verson reports the following
> installed set:
>
> You have installed a subset of VDT version 1.10.1d:
> Berkeley Storage Manager (BeStMan) 2.2.0.10
> CA Certificates v37 (includes IGTF 1.22 CAs)
> Fetch CRL 2.6.6
> Globus Toolkit, web-services, client 4.0.7
> Globus Toolkit, web-services, server 4.0.7
> GPT 3.2autotools2004-NMI-9.0
> GSI-Enabled OpenSSH 4.3
> Grid User Management System (GUMS) Client 1.2.16
> Java 5 SDK 1.5.0_14
> Logrotate 3.7
> MySQL 4.1.22
> MySQL Connector/J 5.0.6
> PRIMA Authorization Module 0.7.1
> PRIMA Authorization Module For GT4 Web Services 0.1.2
>
> Alex: notes problems doing remote gridftp transfers, see below. I
> can log in to the machine directly and see the directories in question
> from a user account, but I see problems even using a simple gridftp
> client like uberftp in accessing any directory:
>
> uberftp> ls
> Could not list ls: 500-Command failed. :
> globus_gridftp_server_file.c:globus_l_gfs_file_stat:389:
> 500-System error in stat: No such file or directory
> 500-A system call failed: No such file or directory
> 500 End.
>
> Looking around the web, I see that this message has been associated
> with GT 4.0.7, e.g. see the following message from Stuart Martin
> (truncated) to the gt-user list in early June:
>
>> Try install the 4.0.7 RFT update package
>> (globus_wsrf_rft_service_java-0.39 ) from http://www.globus.org/toolkit/advisories.html
>> Looks like that could be the problem.
>>
>> -Stu
>>
>> On Jun 5, 2008, at Jun 5, 4:23 PM, Patrick Armstrong wrote:
>>
>>> Hi.
>>>
>>> I've just installed Globus 4.0.7 from source on two Scientific
>>> Linux 5.1
>>> machines, and am having a strange problem when staging files back
>>> from
>>> the execution machine. heplw26 is the workstation I am submitting
>>> the
>>> job from, and gridhn is the execution host. My certificate is
>>> mapped to
>>> dev08 on gridhn and patricka on heplw26.
>>>
>>> When I run the following job, I get an error staging back:
>>> [...]
>>> This is what happens when I run the job:
>>>
>>> [patricka@heplw26 Desktop]$ globusrun-ws -submit -s -F
>>> gridhn.phys.uvic.ca -Ft Fork -f testjob.xml
>>> Delegating user credentials...Done.
>>> Submitting job...Done.
>>> Job ID: uuid:8295dc48-3342-11dd-bbae-001e8c04a58b
>>> Termination time: 06/06/2008 21:01 GMT
>>> Current job state: StageIn
>>> Current job state: StageOut
>>> dev08
>>> Current job state: Failed
>>> Destroying job...Done.
>>> Cleaning up any delegated credentials...Done.
>>> globusrun-ws: Job failed: Staging error for RSL element
>>> fileStageOut.
>>> Can't do MLST on non-existing file/dir /home/dev08/testout.txt on
>>> server
>>> heplw26.phys.UVic.CA [Caused by: Server refused performing the
>>> request.
>>> Custom message: Server refused MLST command (error code 1) [Nested
>>> exception message: Custom message: Unexpected reply: 500-Command
>>> failed : System error in stat: No such file or directory
>>> 500-A system call failed: No such file or directory
>>> 500 End.]]
>>>
>>>
>>> The strange thing is, /home/dev08/testout.txt *does* exist on gridhn
>
> Question to the VDT team: have other people reported this problem?
>
> I verified that I get the same problem on our ITB and production OSG
> 1.0 systems also, but only from my Mac OS X version of the VDT
> client. Updating this client to 1.10.1e does not help; I still get
> the same problem. I also verified that I do NOT get the same problem
> when using uberftp from, e.g., my OSG ITB system to my OSG production
> system. So there seems to be some kind of inconsistency.
>
> Is it possible that the 4.0.7 RFT update package
> (globus_wsrf_rft_service_java-0.39 ) from http://www.globus.org/toolkit/advisories.html
> is not uniformly distributed and applied, e.g. in the SRM client and
> VDT client packages?
>
> Thanks,
> Alan
>
> On Jul 14, 2008, at 12:35 PM, Alex Sim wrote:
>
>> Hi Alan,
>>
>> it's still hanging at STOR even if it's in PASV mode.... maybe it's
>> GLOBUS_TCP_SOURCE_RANGE to test?
>> Is my path on sigmorgh okay (/mnt/hep/osg/asim.test.data)?
>>
>> thanks
>>
>>
>> debug: sending command:
>> TYPE I
>> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
>> :
>> 200 Type set to I.
>>
>> debug: sending command:
>> PBSZ 16384
>>
>> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
>> :
>> 200 PBSZ=16384
>>
>> debug: sending command:
>> PASV
>>
>> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
>> :
>> 227 Entering Passive Mode (129,118,104,41,113,234)
>>
>> debug: sending command:
>> STOR //mnt/hep/osg/asim.test.data
>>
>>
>>
>> -- Alex
>> asim at lbl dot gov
>>
>>
>>
>> On 7/13/08 9:16 PM, Alan Sill wrote:
>>> Hi Alex,
>>>
>>> I forgot to set the globus tcp port range in the $VDT_LOCATION/vdt/
>>> etc/vdt-local-setup.(sh,csh) files on sigmorgh after the last
>>> upgrade. This messes up some of globus's call-back functions.
>>>
>>> Try it now and let me know if it works.
>>>
>>> Thanks,
>>> Alan
>>>
>>> On Jul 11, 2008, at 3:58 PM, Alex Sim wrote:
>>>
>>>> do you have any idea on my gridftp hanging to sigmorgh?
>>>> I was trying to put a file into /mnt/hep/osg/asim.test.data.
>>>> it seems authN was okay... and at the STOR, it was hanging....
>>>> thanks
>>>
>>> Alan Sill, Ph.D
>>> TIGRE Senior Scientist, High Performance Computing Center
>>> Adjunct Professor of Physics
>>> TTU
>>>
>>> ====================================================================
>>> : Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
>>> : e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
>>> ====================================================================
>>>
>>>
>>>
>
> Alan Sill, Ph.D
> TIGRE Senior Scientist, High Performance Computing Center
> Adjunct Professor of Physics
> TTU
>
> ====================================================================
> : Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
> : e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
> ====================================================================
CC: Stuart Martin <smartin@mcs.anl.gov>, Alan Sill <Alan.Sill@ttu.edu>, Rajkumar Kettimuthu <kettimut@mcs.anl.gov>
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Tue, 15 Jul 2008 09:20:18 -0500
To: vdt-support@OPENSCIENCEGRID.ORG, John Bresnahan <buzztroll@gmail.com>
From: Stuart Martin <smartin@mcs.anl.gov>
Download (untitled) / with headers
text/plain 7.4k
I think your RFT patch, 123__rft_problem.diff is the same as the
update package, globus_wsrf_rft_service_java-0.39
But John Bresnahan would be the person to say for sure.

Assuming you have the latest source, then it would be best to send
email to gridftp-user (or gt-user which seems to be the catch all) to
get all relevant eye's on this.

-Stu

On Jul 15, 2008, at Jul 15, 4:10 AM, Alain Roy via RT wrote:

> Hi Alan,
>
> Thanks for reporting the problem. I've added Stu Martin to this
> ticket.
>
> Stu--is Alan Sill's analysis (below) correct? I thought the VDT had
> the important bits of the RFT patch:
>
> http://vdt.cs.wisc.edu/patches/1.10.1/
>
> Are we missing something that we should have?
>
> Thanks,
> -alain
>
> -----------------------------------------------------------------
> Alain Roy vdt-support@opensciencegrid.org
> VDT Support http://vdt.cs.wisc.edu/support.html
>
>> Note to VDT team: we seem to be experinecing problems with a BeStMan
>> installation out of VDT 1.10.1d that points to problems in the
>> gridftp
>> libraries, see below. Running vdt-verson reports the following
>> installed set:
>>
>> You have installed a subset of VDT version 1.10.1d:
>> Berkeley Storage Manager (BeStMan) 2.2.0.10
>> CA Certificates v37 (includes IGTF 1.22 CAs)
>> Fetch CRL 2.6.6
>> Globus Toolkit, web-services, client 4.0.7
>> Globus Toolkit, web-services, server 4.0.7
>> GPT 3.2autotools2004-NMI-9.0
>> GSI-Enabled OpenSSH 4.3
>> Grid User Management System (GUMS) Client 1.2.16
>> Java 5 SDK 1.5.0_14
>> Logrotate 3.7
>> MySQL 4.1.22
>> MySQL Connector/J 5.0.6
>> PRIMA Authorization Module 0.7.1
>> PRIMA Authorization Module For GT4 Web Services 0.1.2
>>
>> Alex: notes problems doing remote gridftp transfers, see below. I
>> can log in to the machine directly and see the directories in
>> question
>> from a user account, but I see problems even using a simple gridftp
>> client like uberftp in accessing any directory:
>>
>> uberftp> ls
>> Could not list ls: 500-Command failed. :
>> globus_gridftp_server_file.c:globus_l_gfs_file_stat:389:
>> 500-System error in stat: No such file or directory
>> 500-A system call failed: No such file or directory
>> 500 End.
>>
>> Looking around the web, I see that this message has been associated
>> with GT 4.0.7, e.g. see the following message from Stuart Martin
>> (truncated) to the gt-user list in early June:
>>
>>> Try install the 4.0.7 RFT update package
>>> (globus_wsrf_rft_service_java-0.39 ) from http://www.globus.org/toolkit/advisories.html
>>> Looks like that could be the problem.
>>>
>>> -Stu
>>>
>>> On Jun 5, 2008, at Jun 5, 4:23 PM, Patrick Armstrong wrote:
>>>
>>>> Hi.
>>>>
>>>> I've just installed Globus 4.0.7 from source on two Scientific
>>>> Linux 5.1
>>>> machines, and am having a strange problem when staging files back
>>>> from
>>>> the execution machine. heplw26 is the workstation I am submitting
>>>> the
>>>> job from, and gridhn is the execution host. My certificate is
>>>> mapped to
>>>> dev08 on gridhn and patricka on heplw26.
>>>>
>>>> When I run the following job, I get an error staging back:
>>>> [...]
>>>> This is what happens when I run the job:
>>>>
>>>> [patricka@heplw26 Desktop]$ globusrun-ws -submit -s -F
>>>> gridhn.phys.uvic.ca -Ft Fork -f testjob.xml
>>>> Delegating user credentials...Done.
>>>> Submitting job...Done.
>>>> Job ID: uuid:8295dc48-3342-11dd-bbae-001e8c04a58b
>>>> Termination time: 06/06/2008 21:01 GMT
>>>> Current job state: StageIn
>>>> Current job state: StageOut
>>>> dev08
>>>> Current job state: Failed
>>>> Destroying job...Done.
>>>> Cleaning up any delegated credentials...Done.
>>>> globusrun-ws: Job failed: Staging error for RSL element
>>>> fileStageOut.
>>>> Can't do MLST on non-existing file/dir /home/dev08/testout.txt on
>>>> server
>>>> heplw26.phys.UVic.CA [Caused by: Server refused performing the
>>>> request.
>>>> Custom message: Server refused MLST command (error code 1) [Nested
>>>> exception message: Custom message: Unexpected reply: 500-Command
>>>> failed : System error in stat: No such file or directory
>>>> 500-A system call failed: No such file or directory
>>>> 500 End.]]
>>>>
>>>>
>>>> The strange thing is, /home/dev08/testout.txt *does* exist on
>>>> gridhn
>>
>> Question to the VDT team: have other people reported this problem?
>>
>> I verified that I get the same problem on our ITB and production OSG
>> 1.0 systems also, but only from my Mac OS X version of the VDT
>> client. Updating this client to 1.10.1e does not help; I still get
>> the same problem. I also verified that I do NOT get the same problem
>> when using uberftp from, e.g., my OSG ITB system to my OSG production
>> system. So there seems to be some kind of inconsistency.
>>
>> Is it possible that the 4.0.7 RFT update package
>> (globus_wsrf_rft_service_java-0.39 ) from http://www.globus.org/toolkit/advisories.html
>> is not uniformly distributed and applied, e.g. in the SRM client and
>> VDT client packages?
>>
>> Thanks,
>> Alan
>>
>> On Jul 14, 2008, at 12:35 PM, Alex Sim wrote:
>>
>>> Hi Alan,
>>>
>>> it's still hanging at STOR even if it's in PASV mode.... maybe it's
>>> GLOBUS_TCP_SOURCE_RANGE to test?
>>> Is my path on sigmorgh okay (/mnt/hep/osg/asim.test.data)?
>>>
>>> thanks
>>>
>>>
>>> debug: sending command:
>>> TYPE I
>>> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
>>> :
>>> 200 Type set to I.
>>>
>>> debug: sending command:
>>> PBSZ 16384
>>>
>>> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
>>> :
>>> 200 PBSZ=16384
>>>
>>> debug: sending command:
>>> PASV
>>>
>>> debug: response from gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data
>>> :
>>> 227 Entering Passive Mode (129,118,104,41,113,234)
>>>
>>> debug: sending command:
>>> STOR //mnt/hep/osg/asim.test.data
>>>
>>>
>>>
>>> -- Alex
>>> asim at lbl dot gov
>>>
>>>
>>>
>>> On 7/13/08 9:16 PM, Alan Sill wrote:
>>>> Hi Alex,
>>>>
>>>> I forgot to set the globus tcp port range in the $VDT_LOCATION/vdt/
>>>> etc/vdt-local-setup.(sh,csh) files on sigmorgh after the last
>>>> upgrade. This messes up some of globus's call-back functions.
>>>>
>>>> Try it now and let me know if it works.
>>>>
>>>> Thanks,
>>>> Alan
>>>>
>>>> On Jul 11, 2008, at 3:58 PM, Alex Sim wrote:
>>>>
>>>>> do you have any idea on my gridftp hanging to sigmorgh?
>>>>> I was trying to put a file into /mnt/hep/osg/asim.test.data.
>>>>> it seems authN was okay... and at the STOR, it was hanging....
>>>>> thanks
>>>>
>>>> Alan Sill, Ph.D
>>>> TIGRE Senior Scientist, High Performance Computing Center
>>>> Adjunct Professor of Physics
>>>> TTU
>>>>
>>>> =
>>>> ===================================================================
>>>> : Alan Sill, Texas Tech University Office: Admin 233, MS
>>>> 4-1167 :
>>>> : e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax
>>>> 806-742-4358 :
>>>> =
>>>> ===================================================================
>>>>
>>>>
>>>>
>>
>> Alan Sill, Ph.D
>> TIGRE Senior Scientist, High Performance Computing Center
>> Adjunct Professor of Physics
>> TTU
>>
>> ====================================================================
>> : Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
>> : e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
>> ====================================================================
>
>
> --
> View ticket at <http://vdt.cs.wisc.edu/rt/Ticket/Display.html?user=guest&pass=guest&id=3671
> >
> VDT Support, vdt-support@ivdgl.org
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Tue, 15 Jul 2008 21:27:48 +0200
To: vdt-support@OPENSCIENCEGRID.ORG
From: Alain Roy <roy@cs.wisc.edu>
Download (untitled) / with headers
text/plain 508b
On Jul 15, 2008, at 4:23 PM, Stuart Martin via RT wrote:
> I think your RFT patch, 123__rft_problem.diff is the same as the
> update package, globus_wsrf_rft_service_java-0.39
> But John Bresnahan would be the person to say for sure.
>
> Assuming you have the latest source, then it would be best to send
> email to gridftp-user (or gt-user which seems to be the catch all) to
> get all relevant eye's on this.

Thanks, I've sent email to gt-user because I'm subscribed to it, and
I'll monitor it.

-alain
CC: Alan Sill <Alan.Sill@ttu.edu>, smartin@mcs.anl.gov
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Tue, 15 Jul 2008 14:50:47 -0500
To: vdt-support@OPENSCIENCEGRID.ORG
From: Alan Sill <Alan.Sill@ttu.edu>
Download (untitled) / with headers
text/plain 1.6k
Interestingly, I do NOT get the indicated error when trying a gridftp
connection from one Linux OSG 1.0 installation to another, but do get
the error when trying the same from my Mac OS X VDT client (VDT
1.10.1e):

uberftp> ls
Could not list ls: 500-Command failed. :
globus_gridftp_server_file.c:globus_l_gfs_file_stat:389:
500-System error in stat: No such file or directory
500-A system call failed: No such file or directory

How sure are we that the patch has been applied to all platforms and
packages that might be affected by it?

Same question for globus_gridftp_server_control-0.21 in http://www.globus.org/toolkit/advisories.html
-- the usernames are in fact longer than 8 characters for the
mappings in question on this system.

Alan

On Jul 15, 2008, at 2:36 PM, Alain Roy via RT wrote:

> On Jul 15, 2008, at 4:23 PM, Stuart Martin via RT wrote:
>> I think your RFT patch, 123__rft_problem.diff is the same as the
>> update package, globus_wsrf_rft_service_java-0.39
>> But John Bresnahan would be the person to say for sure.
>>
>> Assuming you have the latest source, then it would be best to send
>> email to gridftp-user (or gt-user which seems to be the catch all) to
>> get all relevant eye's on this.
>
> Thanks, I've sent email to gt-user because I'm subscribed to it, and
> I'll monitor it.

Alan Sill, Ph.D
TIGRE Senior Scientist, High Performance Computing Center
Adjunct Professor of Physics
TTU

====================================================================
: Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
: e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
====================================================================
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Wed, 16 Jul 2008 11:59:19 +0200
To: vdt-support@OPENSCIENCEGRID.ORG
From: Alain Roy <roy@cs.wisc.edu>
Download (untitled) / with headers
text/plain 1.8k
On Jul 15, 2008, at 9:55 PM, Alan.Sill@ttu.edu via RT wrote:
> How sure are we that the patch has been applied to all platforms and
> packages that might be affected by it?

Quite sure, though I suppose there is a chance of an error.

> Same question for globus_gridftp_server_control-0.21 in http://www.globus.org/toolkit/advisories.html
> -- the usernames are in fact longer than 8 characters for the
> mappings in question on this system.

That's what Charles Bacon pointed out on the gt-user mailing list:

> From: Charles Bacon <bacon@mcs.anl.gov>
> Date: July 15, 2008 10:21:42 PM GMT+02:00
> To: Alain Roy <roy@cs.wisc.edu>
> Cc: gt-user <gt-user@globus.org>
> Subject: Re: [gt-user] GridFTP problem (VDT 1.10.1, GT 4.0.7)
>
> There is no way an RFT update would be affecting uberftp's
> behavior. The only candidate advisory to try is
> globus_gridftp_server_control-0.21, which fixes bug 6066, an issue
> with long user and/or group names. The symptoms don't match up,
> though.
>
> Does globus-url-copy work? Try the GridFTP server troubleshooting: http://www.globus.org/toolkit/docs/4.0/data/gridftp/admin-index.html#s-gridftp-admin-troubleshooting
>
>
> Charles

However, that bug caused a segfault, so it's not clearly the same
problem. That said, we don't have that patch.

Can you try two things?

1) Can you temporarily map to another user with a shorter username?

2) Can you try globus-url-copy as Charles suggested?

This will help us see if the patch might help, or if it's a problem
due to the UberFTP client.

Thanks!

-alain

p.s. I'm traveling in Europe right now, which causes some time lag in
my responses. Sorry for the inconvenience.

p.p.s. What is sigmorgh?

-----------------------------------------------------------------
Alain Roy vdt-support@opensciencegrid.org
VDT Support http://vdt.cs.wisc.edu/support.html
CC: "SRM Support @ LBNL" <srm@lbl.gov>, vdt-support <vdt-support@OPENSCIENCEGRID.ORG>
Subject: Re: sigmorgh gsiftp
Date: Mon, 21 Jul 2008 10:13:11 -0700
To: Alan Sill <Alan.Sill@ttu.edu>
From: Alex Sim <asim@lbl.gov>
Download (untitled) / with headers
text/plain 6.3k
Has this been resolved?
thanks

-- Alex
asim at lbl dot gov



On 7/14/08 11:38 AM, Alan Sill wrote:
> Note to VDT team: we seem to be experinecing problems with a BeStMan
> installation out of VDT 1.10.1d that points to problems in the gridftp
> libraries, see below. Running vdt-verson reports the following
> installed set:
>
> You have installed a subset of VDT version 1.10.1d:
> Berkeley Storage Manager (BeStMan) 2.2.0.10
> CA Certificates v37 (includes IGTF 1.22 CAs)
> Fetch CRL 2.6.6
> Globus Toolkit, web-services, client 4.0.7
> Globus Toolkit, web-services, server 4.0.7
> GPT 3.2autotools2004-NMI-9.0
> GSI-Enabled OpenSSH 4.3
> Grid User Management System (GUMS) Client 1.2.16
> Java 5 SDK 1.5.0_14
> Logrotate 3.7
> MySQL 4.1.22
> MySQL Connector/J 5.0.6
> PRIMA Authorization Module 0.7.1
> PRIMA Authorization Module For GT4 Web Services 0.1.2
>
> Alex: notes problems doing remote gridftp transfers, see below. I
> can log in to the machine directly and see the directories in question
> from a user account, but I see problems even using a simple gridftp
> client like uberftp in accessing any directory:
>
> uberftp> ls
> Could not list ls: 500-Command failed. :
> globus_gridftp_server_file.c:globus_l_gfs_file_stat:389:
> 500-System error in stat: No such file or directory
> 500-A system call failed: No such file or directory
> 500 End.
>
> Looking around the web, I see that this message has been associated
> with GT 4.0.7, e.g. see the following message from Stuart Martin
> (truncated) to the gt-user list in early June:
>
>> Try install the 4.0.7 RFT update package
>> (globus_wsrf_rft_service_java-0.39 ) from
>> http://www.globus.org/toolkit/advisories.html
>> Looks like that could be the problem.
>>
>> -Stu
>>
>> On Jun 5, 2008, at Jun 5, 4:23 PM, Patrick Armstrong wrote:
>>
>>> Hi.
>>>
>>> I've just installed Globus 4.0.7 from source on two Scientific Linux
>>> 5.1
>>> machines, and am having a strange problem when staging files back from
>>> the execution machine. heplw26 is the workstation I am submitting the
>>> job from, and gridhn is the execution host. My certificate is mapped to
>>> dev08 on gridhn and patricka on heplw26.
>>>
>>> When I run the following job, I get an error staging back:
>>> [...]
>>> This is what happens when I run the job:
>>>
>>> [patricka@heplw26 Desktop]$ globusrun-ws -submit -s -F
>>> gridhn.phys.uvic.ca -Ft Fork -f testjob.xml
>>> Delegating user credentials...Done.
>>> Submitting job...Done.
>>> Job ID: uuid:8295dc48-3342-11dd-bbae-001e8c04a58b
>>> Termination time: 06/06/2008 21:01 GMT
>>> Current job state: StageIn
>>> Current job state: StageOut
>>> dev08
>>> Current job state: Failed
>>> Destroying job...Done.
>>> Cleaning up any delegated credentials...Done.
>>> globusrun-ws: Job failed: Staging error for RSL element fileStageOut.
>>> Can't do MLST on non-existing file/dir /home/dev08/testout.txt on
>>> server
>>> heplw26.phys.UVic.CA [Caused by: Server refused performing the request.
>>> Custom message: Server refused MLST command (error code 1) [Nested
>>> exception message: Custom message: Unexpected reply: 500-Command
>>> failed : System error in stat: No such file or directory
>>> 500-A system call failed: No such file or directory
>>> 500 End.]]
>>>
>>>
>>> The strange thing is, /home/dev08/testout.txt *does* exist on gridhn
>
> Question to the VDT team: have other people reported this problem?
>
> I verified that I get the same problem on our ITB and production OSG
> 1.0 systems also, but only from my Mac OS X version of the VDT
> client. Updating this client to 1.10.1e does not help; I still get
> the same problem. I also verified that I do NOT get the same problem
> when using uberftp from, e.g., my OSG ITB system to my OSG production
> system. So there seems to be some kind of inconsistency.
>
> Is it possible that the 4.0.7 RFT update package
> (globus_wsrf_rft_service_java-0.39 ) from
> http://www.globus.org/toolkit/advisories.html is not uniformly
> distributed and applied, e.g. in the SRM client and VDT client packages?
>
> Thanks,
> Alan
>
> On Jul 14, 2008, at 12:35 PM, Alex Sim wrote:
>
>> Hi Alan,
>>
>> it's still hanging at STOR even if it's in PASV mode.... maybe it's
>> GLOBUS_TCP_SOURCE_RANGE to test?
>> Is my path on sigmorgh okay (/mnt/hep/osg/asim.test.data)?
>>
>> thanks
>>
>>
>> debug: sending command:
>> TYPE I
>> debug: response from
>> gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data:
>> 200 Type set to I.
>>
>> debug: sending command:
>> PBSZ 16384
>>
>> debug: response from
>> gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data:
>> 200 PBSZ=16384
>>
>> debug: sending command:
>> PASV
>>
>> debug: response from
>> gsiftp://sigmorgh.hpcc.ttu.edu:2811//mnt/hep/osg/asim.test.data:
>> 227 Entering Passive Mode (129,118,104,41,113,234)
>>
>> debug: sending command:
>> STOR //mnt/hep/osg/asim.test.data
>>
>>
>>
>> -- Alex
>> asim at lbl dot gov
>>
>>
>>
>> On 7/13/08 9:16 PM, Alan Sill wrote:
>>> Hi Alex,
>>>
>>> I forgot to set the globus tcp port range in the
>>> $VDT_LOCATION/vdt/etc/vdt-local-setup.(sh,csh) files on sigmorgh
>>> after the last upgrade. This messes up some of globus's call-back
>>> functions.
>>>
>>> Try it now and let me know if it works.
>>>
>>> Thanks,
>>> Alan
>>>
>>> On Jul 11, 2008, at 3:58 PM, Alex Sim wrote:
>>>
>>>> do you have any idea on my gridftp hanging to sigmorgh?
>>>> I was trying to put a file into /mnt/hep/osg/asim.test.data.
>>>> it seems authN was okay... and at the STOR, it was hanging....
>>>> thanks
>>>
>>> Alan Sill, Ph.D
>>> TIGRE Senior Scientist, High Performance Computing Center
>>> Adjunct Professor of Physics
>>> TTU
>>>
>>> ====================================================================
>>> : Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
>>> : e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
>>> ====================================================================
>>>
>>>
>>>
>
> Alan Sill, Ph.D
> TIGRE Senior Scientist, High Performance Computing Center
> Adjunct Professor of Physics
> TTU
>
> ====================================================================
> : Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
> : e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
> ====================================================================
>
>
>
Download (untitled) / with headers
text/plain 800b
> However, that bug caused a segfault, so it's not clearly the same
> problem. That said, we don't have that patch.
>
> Can you try two things?
>
> 1) Can you temporarily map to another user with a shorter username?
>
> 2) Can you try globus-url-copy as Charles suggested?
>
> This will help us see if the patch might help, or if it's a problem
> due to the UberFTP client.

Alan--

I'm trying to make sure I don't lose this ticket. I don't see a response
from you in the ticket (did you respond outside of the ticket?) so I'm
not sure where it stands.

Did you try these things?

Thanks,
-alain

-----------------------------------------------------------------
Alain Roy vdt-support@opensciencegrid.org
VDT Support http://vdt.cs.wisc.edu/support.html
CC: Alan Sill <alan.sill@ttu.edu>, "ASim@lbl.gov" <ASim@lbl.gov>, "smartin@mcs.anl.gov" <smartin@mcs.anl.gov>
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Thu, 21 Aug 2008 14:59:29 -0500
To: "vdt-support@opensciencegrid.org" <vdt-support@OPENSCIENCEGRID.ORG>
From: Alan Sill <alan.sill@ttu.edu>
Download (untitled) / with headers
text/plain 844b
On Aug 21, 2008, at 11:32 AM, Alain Roy via RT wrote:

> I'm trying to make sure I don't lose this ticket. I don't see a
> response
> from you in the ticket (did you respond outside of the ticket?) so I'm
> not sure where it stands.
>
>> Can you try two things?
>>
>> 1) Can you temporarily map to another user with a shorter username?
>>
>> 2) Can you try globus-url-copy as Charles suggested?
>

I did not try the suggestions, but the problem seems confined, as far
as I can see, to when I use a gridftp client from my Mac OS X machine
to a server on Linux. It seem to work OK from a Linux client to a
Linux server (e.g. via uberftp). All these installations are based on
VDT 1.10.1 .

I think the most productive thing to check would be whether the Mac OS
X package is really up to date with the rest of the VDT.

Thanks,
Alan
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Thu, 21 Aug 2008 13:14:43 -0700
To: "vdt-support@opensciencegrid.org" <vdt-support@OPENSCIENCEGRID.ORG>
From: Alain Roy <roy@cs.wisc.edu>
Download (untitled) / with headers
text/plain 1.1k
On Aug 21, 2008, at 12:59 PM, Alan Sill wrote:

> On Aug 21, 2008, at 11:32 AM, Alain Roy via RT wrote:
>
>> I'm trying to make sure I don't lose this ticket. I don't see a
>> response
>> from you in the ticket (did you respond outside of the ticket?) so
>> I'm
>> not sure where it stands.
>>
>>> Can you try two things?
>>>
>>> 1) Can you temporarily map to another user with a shorter username?
>>>
>>> 2) Can you try globus-url-copy as Charles suggested?
>>
>
> I did not try the suggestions, but the problem seems confined, as
> far as I can see, to when I use a gridftp client from my Mac OS X
> machine to a server on Linux. It seem to work OK from a Linux
> client to a Linux server (e.g. via uberftp). All these
> installations are based on VDT 1.10.1 .
>
> I think the most productive thing to check would be whether the Mac
> OS X package is really up to date with the rest of the VDT.

The Mac OS X binaries are from the same build as all of our Linux
binaries, and each of the platforms built from the same source code.
So yes, I think it really is up to date with the rest of the VDT.

-alain
CC: Alan Sill <alan.sill@ttu.edu>, "ASim@lbl.gov" <ASim@lbl.gov>, "smartin@mcs.anl.gov" <smartin@mcs.anl.gov>
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Thu, 21 Aug 2008 15:30:16 -0500
To: vdt-support@OPENSCIENCEGRID.ORG, Charles Bacon <bacon@mcs.anl.gov>
From: Alan Sill <alan.sill@ttu.edu>
Download (untitled) / with headers
text/plain 504b
On Aug 21, 2008, at 3:18 PM, Alain Roy via RT wrote:

>>>>
>>>> 1) Can you temporarily map to another user with a shorter username?
>>>>
>>>> 2) Can you try globus-url-copy as Charles suggested?
>>

I can't easily map to another user. I can try a g-u-c if someone can
suggest an instructive example.

Given the fact that it happens only when going from Mac OS X to a
Linux-based VDT 1.10.1 gridftp server, there should be a limited set
of thins to check. Note it works fine Linux to Linux.

Alan
CC: vdt-support@OPENSCIENCEGRID.ORG, "ASim@lbl.gov" <ASim@lbl.gov>, "smartin@mcs.anl.gov" <smartin@mcs.anl.gov>
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Thu, 21 Aug 2008 15:55:52 -0500
To: Alan Sill <Alan.Sill@ttu.edu>
From: Charles Bacon <bacon@mcs.anl.gov>
Download (untitled) / with headers
text/plain 1.3k
From what I can see in the email, this looks an awful lot like http://bugzilla.globus.org/globus/show_bug.cgi?id=6066
which has a 4.0.7 advisory issued for it: http://www.globus.org/toolkit/advisories.html?version=4.0#globus_gridftp_server_control-0.21

This is the only message I have by this subject line, so I'm missing a
lot of context; apologies if that update has already been looked into/
applied.

Otherwise, globus-url-copy syntax is easy: globus-url-copy gsiftp://
serv1:port/path/to/src gsiftp://serv2:port/path/to/dest

That does a third-party gridftp transfer of /path/to/src from serv1
to /path/to/dest on serv2. You can replace either side of that with file:///path/to/file
if you want to transfer to or from a local file.



Charles

On Aug 21, 2008, at 3:30 PM, Alan Sill wrote:

>
> On Aug 21, 2008, at 3:18 PM, Alain Roy via RT wrote:
>
>>>>>
>>>>> 1) Can you temporarily map to another user with a shorter
>>>>> username?
>>>>>
>>>>> 2) Can you try globus-url-copy as Charles suggested?
>>>
>
> I can't easily map to another user. I can try a g-u-c if someone
> can suggest an instructive example.
>
> Given the fact that it happens only when going from Mac OS X to a
> Linux-based VDT 1.10.1 gridftp server, there should be a limited set
> of thins to check. Note it works fine Linux to Linux.
>
> Alan
CC: "ASim@lbl.gov Sim" <ASim@lbl.gov>, "smartin@mcs.anl.gov Martin" <smartin@mcs.anl.gov>, Shreyas Cholia <scholia@lbl.gov>
Subject: Re: [vdt-support #3671] Re: sigmorgh gsiftp
Date: Thu, 21 Aug 2008 16:45:29 -0500
To: "vdt-support@opensciencegrid.org" <vdt-support@OPENSCIENCEGRID.ORG>
From: Alan Sill <alan.sill@ttu.edu>
Download (untitled) / with headers
text/plain 652b
On Aug 21, 2008, at 3:59 PM, Charles Bacon via RT wrote:

> From what I can see in the email, this looks an awful lot like http://bugzilla.globus.org/globus/show_bug.cgi?id=6066
> which has a 4.0.7 advisory issued for it: http://www.globus.org/toolkit/advisories.html?version=4.0#globus_gridftp_server_control-0.21
> ...
> Otherwise, globus-url-copy syntax is easy: globus-url-copy gsiftp://
> serv1:port/path/to/src gsiftp://serv2:port/path/to/dest

g-u-c works from both the Mac and Linux side to a remote Linux server.

uberftp fails from Mac client to Linux server on the ls command after
successfully logging in. Should I check more variants?
Download (untitled) / with headers
text/plain 747b
> g-u-c works from both the Mac and Linux side to a remote Linux server.
>
> uberftp fails from Mac client to Linux server on the ls command after
> successfully logging in. Should I check more variants?

Given that the symptom here (UberFTP failing on the Mac, globus-url-copy
working) is shared with Shreya's report on vdt-discuss, I don't think
there is much more for you to do, unless you relish digging into
UberFTP's innards.

I'll try to duplicate the problem shortly, then work with the UberFTP
developers to solve the problem.

-alain

-----------------------------------------------------------------
Alain Roy vdt-support@opensciencegrid.org
VDT Support http://vdt.cs.wisc.edu/support.html
Download (untitled) / with headers
text/plain 415b
Steps to work on this ticket:

1) Install the VDT on our Mac OS X computer.

2) Run globus-url-copy against a VDT installation on another computer.
Verify that it works.

3) Run UberfTP from the Mac OS X installation against the same server.
Verify that it fails.

4) Spend an hour or so looking for an obvious problem.

5) Report the problem to the UberFTP developers.
http://dims.ncsa.uiuc.edu/set/uberftp/

1. Installed

2. globus-url-copy - Works as expected, both sending a file from mac to linux, and pull from linux to mac.

3. uberftp - Fails  as expected:

uberftp> ls 
Could not list ls: 500-Command failed. : globus_gridftp_server_file.c:globus_l_gfs_file_stat:389:
500-System error in stat: No such file or directory
500-A system call failed: No such file or directory
500 End.
uberftp> put ftp-test-mac
No match for ftp-test-mac
uberftp> get /tmp/ftp-test-rhap5
No match for /tmp/ftp-test-rhap5

Next up: Poking around a bit to see if there are any obvious broken things.


4. No obvious causes have appeared. This is the simple globus-gridftp-server.  I can turn off everything in the server install except the ftp server, and it still behaves as described above (g-u-c is always happy. With uberftp, Linux->Linux is okay, MacOSX->Linux fails.)  While I'm not clear on the code reuse in this case, but I don't think RFT (and thus any patches) would impact this.

Reported upstream.  Updates as I get them.

There is a 2.0 release of uberftp, a pretty substantial rewrite from the description.  However, its ChangeLog does not mention this issue, and 2.0 is marked as a beta release.

Upstream says these are two different bugs, both of which are fixed in 2.0.

 

Upstream's message, for context:

The LIST problem this user is getting is most likely due to how uberFTP
uses getopt() in that version. I thought I had fixed this bug in version
1.23 but apparently not.

The failed GET command is due to the /tmp symlink. The 1.2x versions
parsed LIST output which was always problematic. The 2.0 version now
uses MLSX instead which should be less troublesome.

Neither of these bugs surprised me. These two plus others are what
prompted me to write 2.0. BTW it is no longer in beta, that was a mis
print. I hate to do the sterotypical developer thing and just point to
the latest version but you guys should really try it out. It has less
dependencies, more stability and more functionality.

Let me know how things work out. I *might* be able to fix the 'LIST ls'
problem but I highly doubt I'll have enough time to fix the LIST parsing
issues. Actually, 2.0 IS the fix for both of those problems.

CC: Alan Sill <alan.sill@ttu.edu>, "ASim@lbl.gov Sim" <ASim@lbl.gov>, "smartin@mcs.anl.gov Martin" <smartin@mcs.anl.gov>, Shreyas Cholia <scholia@lbl.gov>
Subject: Re: [vdt-support #3671] Uberftp version problems [was: sigmorgh gsiftp]
Date: Thu, 28 Aug 2008 11:34:08 -0500
To: "vdt-support@opensciencegrid.org" <vdt-support@OPENSCIENCEGRID.ORG>
From: Alan Sill <alan.sill@ttu.edu>
Download (untitled) / with headers
text/plain 378b
Thanks for the info. If testing would be helpful, we would be happy
to supply it from here.

The good news is that it would appear not to affect any other VDT
products, if the problems fixed to repair uberftp are not generic.

Alan

On Aug 28, 2008, at 11:19 AM, Alan De Smet via RT wrote:

> Upstream says these are two different bugs, both of which are fixed
> in 2.0.

> If testing would be helpful, we would be happy
> to supply it from here.

It will be helpful, thank you very much for the offer.  I'll let you know when I have binaries to test.

This is a major revision to UberFTP.  Reviewing the ChangeLog (http://dims.ncsa.uiuc.edu/set/uberftp/Changelog ) it looks like it should be largely compatible with existing scripts/documentation using UberFTP.  We're going to check with the author to see if there are some unexpected interface changes.  But given the currently available information, do you anticipate that upgrading to 2.0 will cause any problems?

Subject: Re: [vdt-support #3671] Uberftp version problems [was: sigmorgh gsiftp]
Date: Thu, 28 Aug 2008 10:01:14 -0700
To: vdt-support@OPENSCIENCEGRID.ORG
From: Alain Roy <roy@cs.wisc.edu>
Download (untitled) / with headers
text/plain 353b
On Aug 28, 2008, at 9:42 AM, Alan.Sill@ttu.edu via RT wrote:
> http://crt.cs.wisc.edu/Ticket/Display.html?id=3671
>
> Thanks for the info. If testing would be helpful, we would be happy
> to supply it from here.

Thanks Alan Sill. Once Alan De Smet gets it into a test cache, we
would love to have you test it.

-Alain (the third one on this ticket)
CC: Alan Sill <alan.sill@ttu.edu>, "ASim@lbl.gov" <ASim@lbl.gov>, "smartin@mcs.anl.gov" <smartin@mcs.anl.gov>
Subject: Re: [vdt-support #3671] Uberftp version problems [was: sigmorgh gsiftp]
Date: Thu, 28 Aug 2008 12:24:35 -0500
To: "vdt-support@opensciencegrid.org" <vdt-support@OPENSCIENCEGRID.ORG>
From: Alan Sill <alan.sill@ttu.edu>
Download (untitled) / with headers
text/plain 570b
On Aug 28, 2008, at 12:04 PM, Alain Roy via RT wrote:

> Thanks Alan Sill. Once Alan De Smet gets it into a test cache, we
> would love to have you test it.
>
> -Alain (the third one on this ticket)

The Ala(i)ns are taking over the world, one bug at a time! Only
problem is, at this rate it will take us a while.

There is no rush as I only use uberftp for testing -- hence the
original misdirection in the ticket title, when both Shreyas and I
thought it was an underlying gridftp or library version bug. I'll be
happy to test unser any conditions.

Alan S.

Alan: We have a release candidate for UberFTP 2.0.  Would you be willing to give it a test drive, perhaps kick the tires, and ideally use it for whatever you normally use it for?

You can find the Mac binaries at http://vdt.cs.wisc.edu/software/uberftp//2.0_VDT-1.10.1/uberftp-client-VDT1.10.99-x86_macos_10.4.tar.gz .  Assuming you have an existing VDT 1.10.1 install, and have sourced setup.sh/csh as appropriate, you should be able to just unpack the tarball in some directory and give it a whirl.  I'd suggest not untarring it on top of your existing install; while that should work it's untested and it would replace your existing UberFTP install.

If you use UberFTP on other platforms and are willing to test those as well for regressions, you can find binaries at http://vdt.cs.wisc.edu/software/uberftp//2.0_VDT-1.10.1/

Thanks for the offer to help!  I'm hoping this revision will work well and that we'll have a working official release soon. Unfortunately it's probably too late for 1.10.1h, but 1.10.1i seems likely.

I'm working with Neha Sharma at FNAL to test UberFTP 2.0 against dCache.
Download (untitled) / with headers
text/plain 126b
Neha got back with final results and reports that 2.0 is happy. I'm
hoping Alan Sill will be able to give it a test as well.
CC: Alan Sill <alan.sill@ttu.edu>, Alex Sim <ASim@lbl.gov>, "smartin@mcs.anl.gov Martin" <smartin@mcs.anl.gov>, Shreyas Cholia <scholia@lbl.gov>
Subject: Re: [vdt-support #3671] uberftp problems (was: Re: sigmorgh gsiftp)
Date: Wed, 17 Sep 2008 12:49:29 -0500
To: "vdt-support@opensciencegrid.org" <vdt-support@OPENSCIENCEGRID.ORG>
From: Alan Sill <alan.sill@ttu.edu>
Download (untitled) / with headers
text/plain 3.5k
Apologies for the long delay.

Testing the new version described below seems to work for me on my Mac
OS X 10.5 laptop when the new binary is used on top of the rest of a
VDT 1.10.1 client installation:

Prep: downloaded and untarred the new stuff into a directory, then set
up the stock VDT client:

ad234:~ alansill$ cd '/Users/alansill/Desktop/globus/bin/'
ad234:bin alansill$ . /usr/local/grid/setup.sh
ad234:bin alansill$ grid-proxy-init
Your identity: /DC=org/DC=doegrids/OU=People/CN=Alan Sill 503049
Enter GRID pass phrase for this identity:
Creating proxy ................................................. Done
Your proxy is valid until: Wed Sep 17 23:58:33 2008

Tested old version to verify that it fails:

ad234:bin alansill$ which uberftp
/usr/local/OSG_client_1_0/globus/bin/uberftp
ad234:bin alansill$ uberftp munin.hpcc.ttu.edu
220 munin.hpcc.ttu.edu GridFTP Server 2.5 (gcc64dbg, 1182369948-63)
ready.
230 User tigrepool0010 logged in.
uberftp> uberftp> ls
Could not list ls: 500-Command failed. :
globus_gridftp_server_file.c:globus_l_gfs_file_stat:389:
500-System error in stat: No such file or directory
500-A system call failed: No such file or directory
500 End.
uberftp> quit
221 Goodbye.
kthxbye

Put the new binary into the path, and try again:

ad234:bin alansill$ export PATH=`pwd`:$PATH
ad234:bin alansill$ which uberftp
/Users/alansill/Desktop/globus/bin/uberftp
ad234:bin alansill$ uberftp munin.hpcc.ttu.edu
220 munin.hpcc.ttu.edu GridFTP Server 2.5 (gcc64dbg, 1182369948-63)
ready.
230 User tigrepool0010 logged in.
UberFTP> ls
drwxr-xr-x 12 pool0010 tigre 57344 Aug 21 16:42 .
drwxr-xr-x 4 root root 0 Sep 17 12:01 ..
-rw-r--r-- 1 pool0010 tigre 20594 Jan 29 10:51
gram_job_mgr_32495.log
-rw-r--r-- 1 pool0010 tigre 17766 Jan 30 00:00
gram_job_mgr_19398.log
...
UberFTP> pwd
/home/tigrepool0010
UberFTP> quit
221 Goodbye.


Looks OK!

Any other tests I should do?

Alan



On Sep 2, 2008, at 4:51 PM, Alan De Smet via RT wrote:

> Alan: We have a release candidate for UberFTP 2.0. Would you be
> willing to give
> it a test drive, perhaps kick the tires, and ideally use it for
> whatever you
> normally use it for?
>
> You can find the Mac binaries at
> http://vdt.cs.wisc.edu/software/uberftp//2.0_VDT-1.10.1/uberftp-client-VDT1.10.99-x86_macos_10.4.tar.gz
> . Assuming you have an existing VDT 1.10.1 install, and have sourced
> setup.sh/csh as appropriate, you should be able to just unpack the
> tarball in
> some directory and give it a whirl. I'd suggest not untarring it on
> top of your
> existing install; while that should work it's untested and it would
> replace
> your existing UberFTP install.
>
> If you use UberFTP on other platforms and are willing to test those
> as well for
> regressions, you can find binaries at
> http://vdt.cs.wisc.edu/software/uberftp//2.0_VDT-1.10.1/
>
> Thanks for the offer to help! I'm hoping this revision will work
> well and that
> we'll have a working official release soon. Unfortunately it's
> probably too
> late for 1.10.1h, but 1.10.1i seems likely.
>
>
> --
> View ticket at <http://crt.cs.wisc.edu/Ticket/Display.html?user=guest&pass=guest&id=3671
> >
> VDT Support: vdt-support@opensciencegrid.org

Alan Sill, Ph.D
Senior Scientist, High Performance Computing Center
Adjunct Professor of Physics
TTU

====================================================================
: Alan Sill, Texas Tech University Office: Admin 233, MS 4-1167 :
: e-mail: Alan.Sill@ttu.edu ph. 806-742-4350 fax 806-742-4358 :
====================================================================
Subject: Re: [vdt-support #3671] uberftp problems (was: Re: sigmorgh gsiftp)
Date: Wed, 17 Sep 2008 16:12:36 -0500
To: "Alan.Sill@ttu.edu via RT" <vdt-support@OPENSCIENCEGRID.ORG>
From: Alan De Smet <adesmet@cs.wisc.edu>
Download (untitled) / with headers
text/plain 437b
Alan.Sill@ttu.edu via RT <vdt-support@opensciencegrid.org> wrote:
> Any other tests I should do?

Ideally, "Whatever you normally do with UberFTP that caused you
to notice the problem in the first place." If that's not
practical, could you test that you can cd, get, and put?

Thanks for your time!

--
Alan De Smet Condor Project Research
adesmet@cs.wisc.edu http://www.cs.wisc.edu/condor/
Subject: Re: [vdt-support #3671] uberftp problems (was: Re: sigmorgh gsiftp)
Date: Mon, 22 Sep 2008 18:05:40 -0500
To: "Alan.Sill@ttu.edu via RT" <vdt-support@OPENSCIENCEGRID.ORG>
From: Alan De Smet <adesmet@cs.wisc.edu>
Download (untitled) / with headers
text/plain 337b
As far as we can tell, UberFTP is ready to deploy and fixes the
reported bugs. Unless Alan Sill or others report problems, we'll
include it in the next VDT update, which may be as soon as this
week.

--
Alan De Smet Condor Project Research
adesmet@cs.wisc.edu http://www.cs.wisc.edu/condor/
Subject: [vdt-support #3671] SVN commit, rev 8092
To: vdt-support@cs.wisc.edu
From: adesmet@cs.wisc.edu
Download (untitled) / with headers
text/plain 412b
Commit comment:

1.10.1k will contain UberFTP 2.0, which appears to work as expected.



Changed files:
U vdt/branches/vdt-1.10.1/UberFTP/nmi/build-uberftp
U vdt/branches/vdt-1.10.1/UberFTP/nmi/glue.in
A vdt/branches/vdt-1.10.1/UberFTP/nmi/macosx-fix.patch
U vdt/branches/vdt-1.10.1/UberFTP/nmi/uberftp.in
U vdt/branches/vdt-1.10.1/defs

To generate a diff:
svn diff -c 8092 file:///p/vdt/workspace/svn
Will be in 1.10.1k