-
Notifications
You must be signed in to change notification settings - Fork 8
/
Copy pathRelease_Notes
1253 lines (626 loc) · 120 KB
/
Release_Notes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<html><body><pre>
ENCP release notes, from ENSTORE_RPM_1_0_1_11 to production
Encp changes:
=============
Misc.:
======
Detailed cvs commit logs
========== duplication_util.py ====================================================================================
Replaced calls to pnfs.Pnfs.get_path() with those to find_pnfs_file.find_pnfsid_path().
========== library_manager_client.py ====================================================================================
Changed the incorrect "media changer" label to "library manager".
========== encp_wrapper.py ====================================================================================
Support passing any error messages up to the migration.
Set the encp interface value migration_or_duplication to true. This will allow for some checks to be skipped that the migration has already done.
========== mover.py ====================================================================================
Fixed bugzilla bug 75. It was not fixed in tye previous version as stated
Fixed bugzilla bug 75. Do not write fm if last operation was "read" and the new is "write" for the new volume - HIPRI request.
type fixed
more diagnostics added
include fm writing time into drive time
added diagnostic message to debug bug 75 in bugzilla
modified default timeouts in LM and mover to synchronize them. Bugzilla bug 76
Fixed a bug reported as bugzilla bug #72. There were not enough arguments in the diagnostic message and thread was crashing.
force set null wrapper for disk mover
========== library_manager.py ====================================================================================
modified default timeouts in LM and mover to synchronize them. Bugzilla bug 76
corrected the indentation in mover_error
always update lm part of the ticket so that if lm address changes it gets midified in the ticket
========== duplicate.py ====================================================================================
Made the duplication compatible with the migration code again.
========== enstore_functions3.py ====================================================================================
Added strip_brand() function that returns the opposite information that extrace_brand() does.
========== migrate.py ====================================================================================
Enabled --threading to be passed to encp. Get error messages from encp now that encp.py and encp_wrapper.py support doing so. Made compatibilty (and code cleanup) changes between migration and duplication.
For restored volumes, call vcc.touch() so inventory updates this volume.
Make restarted migration handle swapping after being interupted while swapping.
Fixed --scan-volumes to scan more than one volume at a time. A return statement was in the wrong place.
Made a number of improvments in performace. Replaced pnfs.Pnfs.get_path() calls with find_pnfs_file.find_pnfsid_path() calls. Attempts to reduce the number of db and pnfs queries.
========== encp.py ====================================================================================
Modify do_work() so that it can be used from encp_wrapper.py instead of main().
bumping version to v3_7b because of encpCut
Handle non-retirable errors returned by the mover correctly to stop trying to continue to transfer files. The known issue was when a volume had its wrapper set incorrectly, causing the mover to become confused, which in turn it did not help that encp didn't stop with the retries.
The sequential filenames used by get have changed from <location_cookie> to <volume>:<location_cookie>.
Fixed the errors returned when writing and the givin output path is a regular file. Previously, the input file was wrongly listed, now the output directory is correctly reported.
This patch allows for encp to handle errors returned from the LM when submitting requests. It was possible for a resubmited request to not have the completion_status set to FAILER when a non-retriable error occured.
Added more timing diagnostics to open_control_socket().
Bug fix in verify_file_size() when --override-deleted is used to read a deleted file. p.get_stat() was used, but it has a side effect to return the stats of the directory when the target file does not exist. Now, os.stat() is used directly.
Fix a bug that causes encp on both reads and writes to exit early after a retriable error. This bug was introduced when finish_request() was started to be used by encp, get and put.
Handle a possible tracback in handle_retries(). If the mover closes the control socket, getsockopt() raises an exception that previously was not caught. Also, adjusted the if statement for detecting when to read from standard in; the cron environment makes standard in a pipe, which confused encp. Encp now checks if input files are on the command line. If they are there, then the listed files are used.
Comment out the ability for writes to tape to have standard in as there source. This new functionality was broken for writes running in cronjobs.
Fixed --check. It would check if the LM was ready, but would still continue with the transfer instead of quiting.
Cleanup code. Use constants for Trace.message() calls mostly.
Added support for reading the input from standard in. It is supported on systems that have the /dev/fd/ file system.
Cleaned up the get and encp code. This made for an easier creation of put.py.
More scrubing between encp.py and get.py. Also, started work on the writing parts for the future put command.
Modified encp and get to reduce the amount of duplicate code the two have.
Minor code cleanup between encp.py and get. Some get specific things were removed from encp.py (and moved to get.py).
Just some code cleanup. Nothing functional changed.
1) Use enstore_functions3 module functions instead of local ones. 2) Modified verify_read_request_consistancy() to include checking the CRC and size in layer 2 *before* doing the transfer. Previously, it was only checked *after* the transfer which wasted resources if there was a mismatch.
Added more debug logging for each seperate CRC comparison/check. Previously, only one was done that included all the various CRC checks.
Include a way for processes like encp to state the logging of thread names via their Interface class.
Send encp timing information to the debug log.
bumping version to v3_7 because of encpCut
Allow FQDN in the disk volume labels. For charset, hostnamecharset is now defined.
On Solaris 10 /tmp is a swap partition. os.pathconf(PC_FILESIZEBITS) returns -1 instead of the number of bits. In this case use 32 for the number of bits that can store the size of a file.
Include the user_level (ADMIN, USER or USER2) in the tickets for possible future use.
Fixed the bug in encp that was preventing the RESUBMITS count from being incremented correctly.
When --get-bfid and --override-deleted are test filename to be longer than 0.
When sending an original request (with more copies to follow) update the work_ticket to include the number of copies still to come.
Modified the "encp aborted from" output to include more info.
The patch fixes a bug if --get-bfid matches to to many files for the pnfsid. It uses the bfid in layer 1 to determine which one it really wants.
Use getattr(errno, 'EFSCORRUPTED', errno.EIO) instead of errno.EFSCORRUPTED to avoid pychecker warnings. Added a check in librarysize_check() to check the result of the returned library info from the config server. Currently, goes stait into pulling out data, when the error saying the library does not exist is ignored.
Report a more accurate "Trouble with pnfs" error message in set_pnfs_settings.
Corrected the /etc/mnttab filename for SunOS. Was previously /etc/mntab, which is the wrong name. Also, corrected the Trace.log() line that is supposed to output this error.
Fixed a bug regarding errno module. It should have been errno.EIO and not errno["EIO"].
bumping version to v3_6i because of encpCut
bumping version to v3_6h because of encpCut
Added support for "encp --copy N" where N is the number of extra copies to make of the file. Also, included is a regression test.
Fix encp.py for "get" to work correctly if --sequential-filenames is used.
Support handling 0 or 1 seeded adler32 CRC values instead of just 0 seeded adler32 values.
Fixed some bug that the regression testing found with the previous commit.
Better dealing with metadata when a single machine mounts multiple pnfs servers. This goes for both of encp when reading and for enstore pnfs --path.
Fix the anoying bug that causes sdsscp/get to create new copies of files that have already been read and their metadata created.
bumping version to v3_6g because of encpCut
Address the error handling when encp finds just whitespace located in pnfs layers. It didn't help that encp would set layers to a single space in clear_layers_1_and_4() (also fixed).
bumping version to v3_6f because of encpCut
Change --file-family-wrapper from and admin option to a user2/dcache option.
Log the exception if it occurs in __is_pnfs_local_path(). This should give a clue to what was the real cause the next time it happens. Patched get_volume_clerk_info() to handle a None value for the returned vc_ticket.
Fix a get specific issue related to the create_read_requests() function split into create_read_requests() and create_read_request().
Make the changes for encp to use the new alarm server functionality to send e-mail based on storage groups.
Fixed a spelling error: uninque =? unique.
Catch sys.stderr.write() errors.
Fixed a pnfs_agent related bug. If encp was told to only use the pnfs_agent for PNFS access, then encp was re-throwing the OSError "Force use of pnfs_agent". This re-raise would cause a traceback.
Do the same for create_write_requests() that was done for create_read_requests(). There are now create_write_requests() and create_write_request() functions. This is to provide better error reporting.
Split create_read_requests() into to functions. The new function name is create_read_request(). This is to facilitate better error messages.
If an exception occurs sending an error to the accounting server, don't worry about it. Just log the error and move on. Also, removed some code from create_read_requests() when reading layer 1. This functionality was moved to pnfs.py.
Fixed a bug if the input file is non-existent in a non-pnfs filesystem. Previously an incorrect error message was given. Now the correct one stating that the file does not exist is given.
Fixed is_pnfs_path() to honor REMOTE_ENCP. This was done by using __is_pnfs_remote_path() instead of doing the same (incorrectly) thing inline.
Fixed a bug when reading a deleted file and the user used --override-deleted. The problem files were deleted files that had valid 'pnfs_name0' entires in the Enstore DB. encp was trying to stat() these files and failing the transfer over it.
Fix "get" to work with the new udp_server.py changes.
bumping version to v3_6d because of encpCut
Cleanup of the HSM vs. RHSM file handling.
Protect calls to get_pac() if the pnfs_agent should not be used. This should avoid future false 'pnfs_agent' configuration server errors.
Yet another modification to get_stat(). This one is for it to better take into account the --pnfs-is-automounted switch.
Modified get_stat() to handle intermitent pnfs errors better. This might be new with Linux 2.6, but the EIO errno value can be returned (in addition to or replace ENOENT?). Also, Added a check for the environmental variable REMOTE_ENCP to avoid yet another way to get the 'pnfs_agent' CONFIG error.
Fix a bug where specifying the wrong inputname when reading was resulting in the ".(use)(1)()" file being returned instead of the actuall filename.
bumping version to v3_6c because of encpCut
This should allow for encp to ignore bogus callbacks. This is in response to the security team doing 65,535 port scans. Their scans find encp waiting for a mover to callback and connect. Since the scan does not send what encp is looking for it should ignore the connection. It previously was doing that, but doing so by doing a full retry. Now it goes back to waiting for another connection on the listening socket.
Fix using --get-bfid and --override-deleted switches together. This was a recently introduced bug. Also, catch a situation in handle_retries() to aviod going into an infinit loop when we don't know yet what request attempt failed (aka TCP_EXCEPTION error).
Fixed one more location where the: Got error while trying to obtain configuration: ('KEYERROR', "Configuration Server: no such name: 'pnfs_agent'") errors were comming from.
Fixed a bug in read/write_stall_transfer(). There can be a socket.error raised that wasn't being caught. Now both select.error and socket.error are handled. Also, fixed a bug as to how encp notices if the input file is a directory. Thirdly, fixed a bug that would cause encp to use the wrong tags if the pnfsid contained and "F" because is_access_name() would fail.
bumping version to v3_6b because of encpCut
Support the building of a version of encp for dCache. Workaround for the PNFS ghost file problem. Instead of leaving the true filename as a ghost file, we will leave the temporary filename as a ghost file.
Removed debugging statements.
Missed a global statement that pychecker found.
bumping version to v3_6a because of encpCut
Update encp to allow for regression tests to force encp to use the pnfs agent; even if the specified filesystem is already mounted.
bumping version to v3_6 because of encpCut
Tweek the default behaviour when multiple copies are to be written to tape. Now if the tag has multiple libraries; that number of copies are made. --copies is only needed to override the default count.
Report the os at the start to stdout (if requested) and to the log file. Fix a bug that should allow linux machines to raise alarms for "Filesystem is corrupt" errors. Previously only systems with the EFSCORRUPTED errno would raise an alarm. Removed the write_rhsm_file() function.
Fix traceback bug involving clear_layers_1_and_4().
Reading a file from tape to /dev/null that is bigger than 2GB will fail (at least on linux 2.4). Skip this filesystem test if the output file is: /dev/null, /dev/zero, /dev/random or /dev/urandom. Also, if the input file is /dev/zero, /dev/random or /dev/urandom, then a random number of bytes is transfered. Default bounds are 0 to 2G bytes. ENSTORE_RANDOM_LB and ENSTORE_RANDOM_UB environmental variables can override these default bounds.
bumping version to v3_5e because of encpCut
Modification to the multiple copies feature. First, it is now enabled for users. Second, if you specify more copies than libraries the extra copies are ignored.
Fix using --get-bfid and --override-deleted together.
Rework how the clearing of layer 1 and layer 4 is done when a failure occurs. This should help fix problems with two competing encps.
bumping version to v3_5d because of encpCut
Bug fix for writes. Correctly munge the input file basename if necessary. It was doing it correctly for writes, but not reads.
Removed some debugging print statements that got left in.
bumping version to v3_5c because of encpCut
If the user deleted the entire directory of the pnfs file, don't get confused when trying to re-read the non-existent file-family-width tag. This only applied to writes.
Removed a debugging print.print() call. Fixed a defect in the NOACCES tape error message creating code (tuple usage bug). Lastly, fixed an error with the "media error automatic failover to duplicate copy" code in handle_retries().
Remove a warning about 'pnfs_agent' not being found in the current configuration.
Fixed a bug in handle_retries. On a resubmit, the request_dictionary only contained the "status: (RESUBMITING, None)" value. This was passed to is_read() that did through a traceback. Now, things are rearanged so that the is_read is called only when there is a full request_dictionary. Added to new switches to encp. --mover-timeout and --resubmit-timeout. These allow the (admin) user to adjust these time values.
bumping version to v3_5b because of encpCut
Convert the file_size variable to a long. There is an issue with encp on the 64 bit alphas.
Added a log message after the file is transfered but before waiting for the final dialog.
Fixed a bug. Encp was trying to modify one element of a tuple. It now recreates the entire tuple modifying the one element it wanted to change in the first place.
Multiple copy reads now work. Need to use --get-bfid. Also, fixed EncpError.__init__() to create self.args.
Some adjustments for the 'new' way of doing multiple copies.
Removed a nasty bug from improper tuple unpacking.
Log information that pertains to a recursive loop between handle_retries() and internal_handle_retries().
Use the new atomic.py that does not use delete_at_exit. These changes use delete_at_exit directly for what atomic.py used to do.
Fixed a bug that causes a traceback. verify_file_size() did not pass the filename to encp.Pnfs.get_file_size(). Now it does.
Modification to determine what really caused a traceback in mover_handshake(). Some information is sent to the log file.
Log when encp will clear layers 1 and 4.
Fixed log_encp_start to look for the correct variables in the Interface class. Also, encp will now look at the uid and gid of the output file when doing a dCache write to correctly update the file DB. Otherwise, root.root is put in instead. Lastly, a patch for sdsscp/get (in encp.py) in create_read_requests() so that deleted files can be differentiated from those with partially created metadata.
Oops. Forgot to uncomment something for testing purposes only.
When resending a request grab the current file family width from the tag. If the user overrode the value on the command line, stick with the users value.
Fixed a quoras bug originally reported by John Hendry. If the quota is zero there really is no quota. This fix stops using a zero value as the user having zero quota.
A few more modifications related to getting encp to use the pnfs agent.
Cleanup from previous commit (to get pnfs_agent working with encp).
Merge usage of pnfs.Pnfs and pnfs_agent_client.PnfsAgentClient. The write_to_rshm and read_from_rhsm functions are gone too.
Log the exiting of encp when the user aborts the transfer.
This patch is for multiple copies and the munging of bfids.
Fixed a bug in the previous commit. Moving forward.
Handle a bug in print_data_access_layer(). If socket.gethostbyname() fails to obtain the host information catch the exception. Also, catch socket.gaierror and socket.herror at another use of gethostbyname_ex() and socket.getfqdn().
Made "Filesystem is corrupt" errors alarmable. Encp will fill in the description of a LM request submission NOACCESS/NOTALLOWED error. It will look similar to the message when encp asks the volume clerk directly for the state of the volume.
Emergency bug fix. Attempt to stop encp from looking up the full name of the pnfs target directory when looking for the tags to create the beginning log message. Instead of just doing this for dCache writes with --shortcut it does this for all dCache writes. Also, removed one pnfs.Pnfs() class instantiation from create_write_requests().
remove extraneous print
implemented write via pnfs_agent. this is first iteration needs a lot of work. But it should not break existing code
moving forwatd with writes
introduction of encp reads from remote enstore server pnfs_agent.py and pnfs_agent_client.py get functions from pnfs.Tag class encp.py has new function "read_from_rhsm" and couple of "if"s, new code will not be executed by joe user as it is protected by environment variable "REMOTE_ENCP"
bumping version to v3_5a because of encpCut
Replaced a debugging line of code with the real line that should be there. The debuggin line has the literal "/home/zalokar" instead of the variable request['outfile']. Replaced msg.strerror with str(msg) because the latter gives a better error message (complete with the name of the file that had the problem).
bumping version to v3_5 because of encpCut
Fixed encp so that the --no-crc switch actually works.
This change makes sure that two encps don't clobber each other when writing to tape. Theoretically this collision is only possible with dcache writes, but this change addresses the "what if" it happend form normal writes too.
Fixed a bug with --copies. If --copies was used, there was a problem with the copies failing the newly introduced layer 1 and layer 4 existence. Removed two other debugging print lines that were still in the code.
Added a check in handle_retries() to see if layer 1 and/or layer 4 was set. This check is activated in two places (for writes only): 1) Just before transfer begins. 2) Just before setting the pnfs (and file db) metadata.
When an error comes from EXfer close the data socket. Otherwise we run into a situation where the mover is waiting on the data socket and encp is waiting on the control socket.
Modied encp to use the EXfer module's new quota capability. When a transfer will exeed a users quota the transfer will be aborted. If the transfer will exeed the users soft quota then a warning message will be printed to standard out.
Added support for the encp option --override-deleted. This will allow a user to use --get-bfid and --override-deleted to read any file on a tape.
In the check_crc() function, log the mover CRC and the encp CRC. Previously, only errors were reported. This patch is in responce to a Scientific Linux 3.0.3 with XFS using direct i/o bug. When reading a file larger than 4GB using direct i/o after the 4GB point the kernel/filesystem retrieves the wrong bytes. If it wasn't for a check against the CRC dCache/dccp put into layer 2 we would have stored a corrupted file on tape that would have been missed by CRC checks.
Log the dictionary/ticket passed into is_read() before raising the "Inconsistant file types" exception. This is for debugging purposes.
Reworked how set_pnfs_settings() works for copy transfers. The layers, file size and permsissions should never be changed when a copy is written.
Modified encp to be able to handle using .(access)() filenames on dCache writes. This avoids intensive name lookups to pnfs.
Added a Trace.log() message. In open_control_socket() if a second message is read in this message is logged.
Fixed the --shortcut option for dcache reads. Modified encp to allow --shortcut to work for writes. Added option --override-path to allow the user to specify a path when --shortcut is used. CV: ------------------------------------------
Catch an EncpError that should not happen in the request dictionary was put together correctly.
Fixed a bug that prevented the pnfsid to be set in the filedb for copies.
Added original_bfid to the file clerk subticket. (It should have already been there). Place the copies in the master list of write requests. Before they were stored inside the ticket of the original request. This change was motivated when printing the requests for the original request were being cut off. Fixed a bug that compared the conanical path with a hard coded first 19 characters of the path. 19 works for /pnfs/fnal.gov/usr/ but not in the general case. Now it uses len(conanical_path).
Moved the copy.deepcopy() call to inside the loop when creating the requests for copies. A traceback was occuring with the copy before the loop becuase the file_family was subticket entry was already removed.
Fixed a problem at the beginning to recognize when a file is a pnfs or local file w.r.t the check for all three types of paths. The old way had problems with automounting paths not beginning with /pnfs/... Found a few more KEYERROR vs. NO_VOLUME/NO_FILE bugs. The copy numberi, when --copies is used, is relative to this encp. It is not absolute w.r.t. the number of copies of the file in the entire system. Also, the original bfid and original file family are not mangled on the encp side. They will be mangled on the mover side.
Improved --check to return an exit status of 2 for 'temporary errors'. Also, take advantage of the new TIMEDOUT errors, and NO_VOLUME and NO_BFID. Lastly, place into the accounting db the new fields supported by the accounting client (like file_family, volume, drive, etc.).
Modified --verbose with a level less than zero to also turn off the data access layer for both successes and failures. Also, fixed the "Not all input files are %s files" message to correctly list /pnfs/... or unix.
Fixed two bugs with respect to --data-access-layer. First, was that successful transfers with this option enabled would place the success into the encp_error table in the accounting database. Second, on successes the information printed out to the screen with this option enabled was not printing out the correct information (it was leaving most values blank).
Modified the "Volume clerk (%s) knows nothing about %s.\n" messages. If the volume clerk returned a KEYERROR then the old error is still logged. If a different error is returned then, "Failure communicating with volume clerk..." is logged instead.
Attempt to handle an exception. If the output file did not exist when trying to get the directory, an OSError occured. It is now caught inside of the log_encp_start() function.
Comment out the --copy option for the v3_4 cut.
bumping version to v3_4 because of encpCut
Added the --copy switch for encp. This switch allows for encp to write N copies of the file to different tapes.
Fixed an error reported. Two cases exist. One is when get_file_clerk_info() is passed a ticket and not just the bfid when an error occurs. The other is when get_volume_clerk_info() also has an error when a ticket (and not just the external label) is passed to it. When the error string was put together the entire ticket was used instead of just the bfid or volume.
Fixed e_access() to handle or-ed os.R_OK, os.R_OK and os.X_OK values. Not just one of them at a time.
Use the new code in enstore_functions2.py 1.12 to handle reading filenames like: stkensrv2:7500//pnfs/test/testfile. Also, fixed a bug in reading some bfids. Some 2000 bfids from D0 have "DMS" bfids instead of "D0MS". Encp had problems with these bfids. The brand and bfid handling code has been fixed to handle brands with lengths other than that of four bytes.
Modified print_data_access_layer_format() to check for sub-tickets to actually be dictionaries. There seems to be a way for a string to be named one of these sub-tickets that causes tracebacks.
Use the new accounting server ability to handle None in the message field for log_encp_error().
Modify how encp, ensync and enmv use the option.Interface class's print_usage() function. All command line parsing errors, should now give an error return code.
Don't pass None to log_encp_error() for the error paramter. Pass "" instead.
Fixed the cause of an UNCAUGHT_EXCEPTION. The if...else branch did create a situation where an uninitialized variable could be used.
Bug fix in the way encp handles bogus libraries. If the library specified does not exist, an UNGAUGHT_EXCEPTION would occur. This is now handled.
Fixed a handle_retries bug such that the queue size is now correctly determined on errors. Also, when such error occurs, it is not deleted from the queue since the 'completion_status' ticket field is now used.
It might be possible that an error while reading could be forgetten about when the exit_status is printed/returned. Modified the if...elif...else statement to protect against wrongly falling into the 'else' section.
Encp resends on writes were being done twice instead of just once. This has been fixed. Also, the messages to the user and log file specifing that encp is trying to submit have been "cleaned up" and only occur from one spot in the code now, namely submit_one_request(), instead of all over.
Added "get" option --skip-deleted-files. This option will cause get to skip over deleted files.
Bug fix. Now it handles exceptions raised when the file has been deleted before it attempts to clobber layer 1 & 4 during a dcache write.
Fixed the message string that is created when a volume is NOACCESS (the volume name was listed as the entire work_ticket as a string). Also, modified where print_data_access_layer_format() looks for the volume in the ticket to print the most accurate information. It was not handing the case where only the 'external_label' in the VC sub-ticket was known, but none of the others.
The switch --bypass-filesystem-max-filesize-check was broken. Fixed it.
Added support for encp to record failures in the accounting database. Added the UNCAUGHT_EXCEPTION error to give to users when encp would otherwise traceback.
Fix a "get" bug. When get would attempt to read a file with only a partial entry in the file database and no corresponding pnfs file; the new pnfs file would not be created because the pnfs class instance wasn't getting instantiated correctly. Added a pnfs.Pnfs() call and bug fixed.
Handle an AttributeError exception for "get" when the pnfs file does not exist when entering the check_crc() function. This check is for the dcache layer 2 check.
Fixed use of the --check option. I had become broken.
Fixed two bugs relating to ignoring of the --library and --file-family-wrapper switches. The first was only cosmetic. The First log file message printed the wrong values. The second was due to always looking at the output directory tags when checking the maximum size of the library and file family wrapper.
Modified the control socket error detection section in handle_retries(). It might be eating messages. Now just check if the queue is empty or not after the select().
Modified encp to determine the next transfer similar to how "get" does so. This involves using the 'completion_status' field in the request ticket. This work is done it preperation of having an API (for automigration).
Added CRC_DCACHE_MISMATCH to the list of errors that can be reported from encp. This differentiates this type of error from CRC_ENCP_MISMATCH which was used for both situations.
Modified when/how the control socket is accessed before transfer_file(). There might have been eaten final dialogs for really small files. This entailed two new functions: stall_read_transfer() and stall_write_transfer() that encapsulate mostly existing inline functionality.
Added code to detect (with multiple file transfers) if two input files will be copied to the same output file. (i.e. same name; two input dirs)
I changed the timeout values when checking the control socket for any unexpected problems. Instead of 5 seconds it is now 1 second.
Misspelled max_resumit as max_resubmits.
Removed a comma that mylint.py did not like.
Removed dependance on brands to determine if a file clerk knows about a bfid. Made a change to use the newer configuration_client.py caching mechanism. Started adding accounting_client code to have encp record information about failure to the accounting database. Still commented out. Moved max_attempts() to inside create_read/write_requests(). The retry/resubmit (and maxes) information is now placed inside a sub-ticket. Moved the external_label checks around. This is to avoid a check after each file (which is painful for get). Bug fix for get; input filenames were always getting the sequetial filenames even when the real names were given via --list.
get_csc() can raise exceptions. Some places in encp/get did not catch them. They are now caught.
Handle bfids with "L" appended to them correctly. This would cause an exception and the exception handling code had another bug that caused a traceback. Also, handle files where the file database metadata has None for the crc. This was also causing tracebacks. Lastly, make a preparatory change to EncpError to place the "status" error inside the optional ticket if not already done so.
Fixed two bugs regarding encp not recognizing some bfids. Some older bfids (Nov 2000 were when the sample file's bfids were created) appended a long() "L" to the end of the bfid. This L is consistant between the pnfs metadata and the file database. Thus, encp will now recognize these correctly. The other bug was in the error handling code that tried to use the wrong undefined variable when forming the error string for the first bug mentioned.
Fixed a bug where Pnfs.__init__() would succed, but if get_path() was passed as the parameter the empty string, a traceback would occur. Pnfs.get_path() can now throw ValueErrors (in addition to OSErrors or IOErrors), which are now handled by encp.
Modified the code to deal with some occasional errors. These occur when __get_fcc() (and __get_vcc()) return None for the volume/bfid information. This can occur (correctly) for some cases; like None was passed into the function for the volume/bfid parameter. However, for these error cases there should have been a valid value. These changes should handle the error and simultaneously determine what is actually happening.
bumping version to v3_3b because of encpCut
Modified encp to avoid some uncesessary recreates of account_client class instances. This was anoying in that warning messages can be sent when test systems without accounting servers are used. (Now only one such message will appear.)
Modified __str__() in EncpInterface to return a more useful string for printing. Added more "Time to" messages for debugging messages. In handle_retries() added code to resubscribe to the csc's erc to receive NEWCONFIGFILE messages. Added elapsed() and elapsed_string() functions to aid in the sanity of including "elapsed=..." to the end of terminal messages.
Attempting to fix bugs in get. One is that the handling of NOACCESS errors in the previous encp.py commit is still broken. The other is to add various "Time to" messages throughout encp.py and get.py to find slow code.
Handle the exception raised withing handle_retires() when it attempts to determine if the tape is NOACCESS or NOTALLOWED. The exception would lead to an encp/get traceback.
Added carbage collection code. This is to force collection at otherwise sleeping times while waiting for responces. Modified handle_retries to be able to check the local file (for existance, etc.). This is to be able to better show that encp does not randomly delete files. Modified how final_say and delete_at_exit.quit() work together. This is to better support the automigration project.
Modified these clients (encp and entv) to take advantage of caching items from the configuration. They both attempt to dump the entire configuraton at the beginning. Other changes include minimizing the number of other server queries too.
Fixed how error from open_local_file() get processed. Errors like ENOENT are now classified as the non_retriable e_errors.FILE_MODIFIED error. Also, added a check in open_local_file() to compare the filesize too in addition to the inode.
Small mods this time. Used is_read() where it wasn't used before. Added some comments to the option/switch dictionary. Fixed an exception raised in is_read() if the argument passed in is invalid.
Addressed a traceback that would occur if the library tag (on writes) named a non-existent library. Also, modified the handling of the --max-retry and --max-resubmit command line switches to support understanding the string 'None' as a value.
Added setting the uid and gid in the file database at the end of a write. Also, reset the mover_timeout back to 600 seconds (I had commited a debug change that shank this number).
Cleaned up the udp_server instance usage in open_udp_socket(). It was looping twice (once in open_udp_socket and once outside) for the same information to be recieved. This combined with a bug (now fixed in 1.23) in cleanUDP.select() allowed for twice the specified timeout to be waited.
Added encp admin options --file-family-width and --ignore-storage-group for use with automigration. Fixed a bug in option.py. In some situations it wouldn't parse the command line correctly. Also added an example that shows how to set more than one variable without having any additional arguments.
Added check incase the same input file is listed twice. This used to be done, but I must have removed it at some point.
These changes give encp/get a usable --check option. Most of the changes are for allowing encp/get to turn off the logging of messages (without having explicit if statements throughout the code). Others are simple cleanup changes (from pychecker).
Fixed the error message given when either create_read_requests() or create_write_requests() fails. Previously, the reason given to the user was not very useful.
A bug introduced into encp.py revision 1.680 was that the command line was not being sent to the log server. This is now fixed.
Use the power of the new enroute2 (using arping) to handle the situation where the interfaces are on the same subnet and avoid the wrong arp response problem.
bumping version to v3_3a because of encpCut
encp has been modifed to no longer need the UDP socket to obtain routing information from the mover. The control socket (and data socket) can now survive past a routing change that previously would have rendered it useless; thus the UDP socket hack. Even though get still needs to use the UDP socket, it is now compatible with the changes in encp.
Added the options --storage-group, --file-family-wrapper and --library to encp for use in the automigration project.
Cleaned up pychecker warnings. Note: One change does require callback.py 1.100.
bumping version to v3_3 because of encpCut
Fixed some situations where encp does not print as much information that is known in the data_access_layer_format. Notably: it is not printing the filesize when the sizes do not match when writing from dcache.
Fixed a bug. Copy paste error in transfer_file() from when the bytes_not_transfered fix was done.
bumping version to v3_2 because of encpCut
Added timeout to the get_brand() and get_bfid() file clerk client calls.
Fixed an inconsistancy in the way the bytes_not_transfered was utilized from EXfer.
Commented out a no longer useful log message. Modified the check for looking for inconsistancies between the files pnfs layers and the file database on reads.
Included some information in the exceptions raised when testing for valid filenames, filesystem filesize limitations, library filesize limitations, etc. that includes the filenames for the print_data_access_layer_format().
Modifications to hit pnfs less frequently. There were far more stats happening than needed to be.
When writing to enstore, if the output pnfs file is moved to a new location or renamed then encp will use the pnfs id to find the new location. If the pnfs file is removed, then the behavior is unchanged; the transfer fails.
When an error occurs writing to enstore via dcache, layer 1 and layer 4 are now cleared.
More fixes like in revision 1.657.
Added a pstatinfo() (really a pstatdecode()) call to the verify_file_size() function. This should force the filesize to be decoded before it is used.
At some point the "get" option --sequential-filenames was broken. Now fixed.
Fixed the order things are done for creating the wrapper sub ticket. Mode should come from the local file when writting to enstore, but was getting overridden by the mode for the directory where the pnfs file would be created.
Modified what gets logged when sending request to the LM. Some difference if the error was retriable or non-retriable. If the error was e_errors.MALFORMED then the entire ticket that was sent is logged.
Fixed a holdover bug from previous commit. It did work with /dev/null as the output directory when I tested it, but didn't work with a real directory until now. Added comments on the large if...elif...else statement in create_read_requests() and what to keep in mind when modifying a branch within it.
Fixed a bug for "get". There was a variable that could be referenced before being defined when there was no metadata available.
Modified the code to remove unecessary pnfs queries.
Changed how the default wrapper fields are filled in.
Added some flush() calls to keep the terminal output up to date.
Added a \n to a message to standard out.
Added check to make sure that the fcc.server_address and vcc.server_address values are legit and not None.
Modified the warning/information message given when sdsscp/get reads a tape for the first time when no metadata is known.
Add support for get to have a --sequential-filenames option to override any existing filename metadata and use sequentially numbered filenames instead.
Fixed two bugs related to less than perfect handling of the default enstore system "going away" (i.e. stopped, shutdown, powered off). The first resulted in a confusing warning message. The other resulted in a traceback when trying to resumit a request.
Attempt to handle "connection reset by peer" errors on the data socket with shutdown(). Added the string 'mover' and 'encp' to the crc mismatch error messages. Tweeked transfer_file() for use by get. Added the crc to the MSG_TYPE=ENCP_XFER log messages.
Fixed a bug when the requested volume is not known to any enstore system. Fixed a bug that could cause "get" to miss a renamed file and create a duplicate filename entry.
Fixed a bug in handle_retries. If there were 3 errors and the new "error" was RESUBMITTING it would incorrectly interpret it as a TOO_MANY_RETRIES. It should have remained a RESUBMITTING "error."
Added SystemExit to occurances of KeyboardInterrupt before blanket excepts. Socket creation failures (from callback mostly) are now handled. Volume requests to the volume clerk for None volumes has been stopped.
Fixed a bug that would raise an alarm when fcc.set_pnfsid() fails, but would not abort the transfer. This would result in complete metadata. Only observed in get transfers though. The tags in the "command line" info message on --put-cache writes now display correct info, instead of "unknown". The accounting server is now sent the version when a transfer completes.
Moved a trace message to contain accurate information with respect to location on the tape.
Two fixes. One, one call of tape_ticket remained that tried to turn the tape_ticket['tape_list'] from a dictionary into a list. Newer fcc return this as a list already. Two, stopped using fcc.get_bfids() for volume (aka "get") transfers.
Removed a debugging print statement. Added a raise of KeyError so a warning message will get displayed for "get".
Added one more reason to the comment describing what can cause the pnfs filesystem to be unwriteable. This reason was that the node was not in the trusted list of nodes.
Fixed a bug that caused encp to ask the volume clerk about the volume consiting of the empty string, instead of a string containing the volume name.
This modification handles re-raising exceptions correctly. If you call "raise sys.exc_info()" it does not work (as far as I can tell it should). They have been replaced with: raise sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2] to re-raise the exception without creating tracebacks.
Attempting to fix a problem with a traceback on stkendca8a. A mod for get to correctly handle unknown files.
Lower-level functions won't exit out directly. They pass the error up to higher levels for processing. More cleanup/optimization of the volume with list code in create_read_requests().
Encp now remembers the inode of the file when first stat()ed. This required splitting create_zero_length_files() into create_zero_length_pnfs_files() and create_zero_length_local_files(). verify_file_size() and open_local_file() now have checks in them using inode values. Since all of the direct i/o code is now in EXfer, old/unused code was removed from open_local_file().
Added two new functions, get_vcc() and get_fcc(). They work similar to get_csc() in that a cached global variable is stored after the first call. The access_check() function is placed in more places. This will hopefully reduce automounting pnfs related errors (when --pnfs-is-automounted is used). Handles exceptions raised from select.select().
Last few bugs w.r.t. get working. Added some log messages for the few errors that remain.
Major fixes for get to work: 32767b acc_daily_summary.py accounting_client.py accounting_client.pyc accounting.py accounting_query.py accounting_server.py aci.py aci.py.000 aci.pyc alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.pyc aml2_dummy.py aml2_dummy.py~ aml2_dummy.pyc aml2_log.py aml2.py aml2.py.000 aml2.py.sav analize_queue.py atomic.py atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py bs.py bs.py~ callback.py callback.pyc callback.py.old cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.pyc cl_info.py clnt.py config-filec configuration_client.py configuration_client.py.000 configuration_client.pyc configuration_server.py configuration_server.py.000 configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py dcache_monitor.py delete_at_exit.py delete_at_exit.pyc discard_copy.py discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.pyc drivestat2.py drivestat_client.py drivestat_client.pyc drivestat_server.py drive_utilization_plotter_module.py dumm.py dumm.py~ duplicate.py duplication_util.py ecron_util.py edb.py edb.pyc e_errors.py e_errors.pyc ejournal.py ejournal.pyc encp_admin.py encp.py encp.py.000 encp.pyc encp.py.with_pipe encp_rate_multi_plotter_module.py encp_ticket.py encp_ticket.pyc encp_user2.py encp_user.py encp_wrapper.py en_eval.py en_eval.pyc enmail.py enmv_admin.py enmv.py enmv_user.py enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py ENSTORE_BIN enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.pyc enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files.py enstore_files.pyc enstore_functions2.py enstore_functions2.pyc enstore_functions3.py enstore_functions3.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_pg.py enstore_plots.py enstore_plotter_framework.py enstore_plotter_module.py enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.pyc enstore_saag.py enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_system_html.py enstore_up_down.py enstore_up_down.pyc enstore_user2.py enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py esgdb.py espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_family_analysis_plotter_module.py fileinfo.py file_utils.py find_pnfs_file.py find_pnfs_file.pyc flip_tab_acsls.sh flip_tab_das.sh from_gssensrv2 ftt_driver.py ftt.py gccensrv2 generic_alarm.py generic_client.py generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.pyc get_all_bytes_counter.py GET_BIN get_cron_title.py get_hosts.py get.py GET_TEMP get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.pyc idlemovers.py info_client.py info_client.pyc info_server.py inquisitor.ann inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor.py interface.py inventory.py journal.py label_tape.py library_manager_client.py library_manager_client.py.000 library_manager_client.pyc library_manager_db.py library_manager_db.py~ library_manager.py library_manager.pyc library_manager.py.optimization_work library_manager.py.optimization_work~ library_manager.py.with_purge lintit lm_list.py lm_list.pyc lm_tick lm_tick~ lock_2.py lock_2.py~ lock_os.py lock.py lock.py~ log_client.py log_client.pyc log_finish_event.py log_server.py log_start_event.py log_trans_fail.py log_trans_fail.pyc lsof.py lsof.py~ lustre_namespace.py lustre_namespace.py~ m2.py Makefile Makefile.000 Makefile.sav make_ingest_rates_html_page.py make_migrated_as_duplicate.py make_original_as_duplicate.py makeplot.py make_pyc_files.py manage_queue_db.py manage_queue_db.py~ manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py.000 media_changer.py.1.266 media_changer.pyc media_changer.py.IBM3584 media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client1.py monitor_client1.py~ monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py mounts_plotter_module.py movcmd_mc.py mover1.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover-nanny.pyc mover.py mover.py.0000 mover.py.000001 mover.py.02-29 mover.py.1010 mover.py.1.1000 mover.pyc mover.py.Jun_27 mover.py.sav mpq_1_2.py mpq_1_2.py~ mpq_exper.py mpq.py mpq.pyc mpq.py.sav multiple_interface.py multiple_interface.pyc net_directory.py net_driver.py net_driver.pyc npipe null_driver.py null_wrapper.py on-call.py operation.py option.py option.pyc overr.py plotter_main.py plotter.py pnfs_agent_client.py pnfs_agent_client.pyc pnfs_agent.py pnfs_backup_plot.py pnfs_backup_plotter_module.py pnfsidparser.py pnfs.py pnfs.pyc priority_selector.py priority_selector.pyc put_get_nw.py put_get_nw.py~ put_get_pipe.py put_get_pipe.py~ put_get.py put_get.py~ put_get.py.000 put_get.py.0001 put_get_udp.py put_get_udp.py~ put.py q_t.py quickquota.py quota.py quota.pyc quotas_plotter_module.py ratekeeper_client.py ratekeeper_client.pyc ratekeeper_plotter_module.py ratekeeper.py ratemeter.py rate_test.py readonly_volumes.py recent_file_listing.py Release_Notes report_volume_quotas.py restoredb.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.pyc sde_admin.py SDE.py SDE.py.sav send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py slots_usage_plotter_module.py srv.py stethoscope.py string_driver.py string_driver.pyc summary_burn_rate_plotter_module.py summary_plots.py swap_original_and_copy.py tab_flipping_nanny.py take_out_acsls.sh take_out_das.sh tapes_burn_rate_plotter_module.py tcpip1.py tcpip1.py~ tcpip.py tcpip.py~ test_interval.py time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.pyc tr_test.py tr_test.py~ ttt.py ttt.py~ udp_client.py udp_client.pyc udp_cl.py udp_common.py udp_common.pyc udp.py udp.py~ udp_server.py udp_server.pyc udp_srvr.py vdbdump.py volume_assert.py volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_family.py volume_family.pyc web_server.py weekly_summary_report.py www_server.py www_server.pyc YANKED yank.py Fixed a bug that determines the correct CS knowing the volume. 32767b acc_daily_summary.py accounting_client.py accounting_client.pyc accounting.py accounting_query.py accounting_server.py aci.py aci.py.000 aci.pyc alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.pyc aml2_dummy.py aml2_dummy.py~ aml2_dummy.pyc aml2_log.py aml2.py aml2.py.000 aml2.py.sav analize_queue.py atomic.py atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py bs.py bs.py~ callback.py callback.pyc callback.py.old cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.pyc cl_info.py clnt.py config-filec configuration_client.py configuration_client.py.000 configuration_client.pyc configuration_server.py configuration_server.py.000 configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py dcache_monitor.py delete_at_exit.py delete_at_exit.pyc discard_copy.py discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.pyc drivestat2.py drivestat_client.py drivestat_client.pyc drivestat_server.py drive_utilization_plotter_module.py dumm.py dumm.py~ duplicate.py duplication_util.py ecron_util.py edb.py edb.pyc e_errors.py e_errors.pyc ejournal.py ejournal.pyc encp_admin.py encp.py encp.py.000 encp.pyc encp.py.with_pipe encp_rate_multi_plotter_module.py encp_ticket.py encp_ticket.pyc encp_user2.py encp_user.py encp_wrapper.py en_eval.py en_eval.pyc enmail.py enmv_admin.py enmv.py enmv_user.py enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py ENSTORE_BIN enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.pyc enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files.py enstore_files.pyc enstore_functions2.py enstore_functions2.pyc enstore_functions3.py enstore_functions3.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_pg.py enstore_plots.py enstore_plotter_framework.py enstore_plotter_module.py enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.pyc enstore_saag.py enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_system_html.py enstore_up_down.py enstore_up_down.pyc enstore_user2.py enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py esgdb.py espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_family_analysis_plotter_module.py fileinfo.py file_utils.py find_pnfs_file.py find_pnfs_file.pyc flip_tab_acsls.sh flip_tab_das.sh from_gssensrv2 ftt_driver.py ftt.py gccensrv2 generic_alarm.py generic_client.py generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.pyc get_all_bytes_counter.py GET_BIN get_cron_title.py get_hosts.py get.py GET_TEMP get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.pyc idlemovers.py info_client.py info_client.pyc info_server.py inquisitor.ann inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor.py interface.py inventory.py journal.py label_tape.py library_manager_client.py library_manager_client.py.000 library_manager_client.pyc library_manager_db.py library_manager_db.py~ library_manager.py library_manager.pyc library_manager.py.optimization_work library_manager.py.optimization_work~ library_manager.py.with_purge lintit lm_list.py lm_list.pyc lm_tick lm_tick~ lock_2.py lock_2.py~ lock_os.py lock.py lock.py~ log_client.py log_client.pyc log_finish_event.py log_server.py log_start_event.py log_trans_fail.py log_trans_fail.pyc lsof.py lsof.py~ lustre_namespace.py lustre_namespace.py~ m2.py Makefile Makefile.000 Makefile.sav make_ingest_rates_html_page.py make_migrated_as_duplicate.py make_original_as_duplicate.py makeplot.py make_pyc_files.py manage_queue_db.py manage_queue_db.py~ manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py.000 media_changer.py.1.266 media_changer.pyc media_changer.py.IBM3584 media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client1.py monitor_client1.py~ monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py mounts_plotter_module.py movcmd_mc.py mover1.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover-nanny.pyc mover.py mover.py.0000 mover.py.000001 mover.py.02-29 mover.py.1010 mover.py.1.1000 mover.pyc mover.py.Jun_27 mover.py.sav mpq_1_2.py mpq_1_2.py~ mpq_exper.py mpq.py mpq.pyc mpq.py.sav multiple_interface.py multiple_interface.pyc net_directory.py net_driver.py net_driver.pyc npipe null_driver.py null_wrapper.py on-call.py operation.py option.py option.pyc overr.py plotter_main.py plotter.py pnfs_agent_client.py pnfs_agent_client.pyc pnfs_agent.py pnfs_backup_plot.py pnfs_backup_plotter_module.py pnfsidparser.py pnfs.py pnfs.pyc priority_selector.py priority_selector.pyc put_get_nw.py put_get_nw.py~ put_get_pipe.py put_get_pipe.py~ put_get.py put_get.py~ put_get.py.000 put_get.py.0001 put_get_udp.py put_get_udp.py~ put.py q_t.py quickquota.py quota.py quota.pyc quotas_plotter_module.py ratekeeper_client.py ratekeeper_client.pyc ratekeeper_plotter_module.py ratekeeper.py ratemeter.py rate_test.py readonly_volumes.py recent_file_listing.py Release_Notes report_volume_quotas.py restoredb.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.pyc sde_admin.py SDE.py SDE.py.sav send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py slots_usage_plotter_module.py srv.py stethoscope.py string_driver.py string_driver.pyc summary_burn_rate_plotter_module.py summary_plots.py swap_original_and_copy.py tab_flipping_nanny.py take_out_acsls.sh take_out_das.sh tapes_burn_rate_plotter_module.py tcpip1.py tcpip1.py~ tcpip.py tcpip.py~ test_interval.py time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.pyc tr_test.py tr_test.py~ ttt.py ttt.py~ udp_client.py udp_client.pyc udp_cl.py udp_common.py udp_common.pyc udp.py udp.py~ udp_server.py udp_server.pyc udp_srvr.py vdbdump.py volume_assert.py volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_family.py volume_family.pyc web_server.py weekly_summary_report.py www_server.py www_server.pyc YANKED yank.py Removed costly requests of the file/volume clerk for volume w/ list transfers. It now uses tape_ticket values. 32767b acc_daily_summary.py accounting_client.py accounting_client.pyc accounting.py accounting_query.py accounting_server.py aci.py aci.py.000 aci.pyc alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.pyc aml2_dummy.py aml2_dummy.py~ aml2_dummy.pyc aml2_log.py aml2.py aml2.py.000 aml2.py.sav analize_queue.py atomic.py atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py bs.py bs.py~ callback.py callback.pyc callback.py.old cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.pyc cl_info.py clnt.py config-filec configuration_client.py configuration_client.py.000 configuration_client.pyc configuration_server.py configuration_server.py.000 configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py dcache_monitor.py delete_at_exit.py delete_at_exit.pyc discard_copy.py discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.pyc drivestat2.py drivestat_client.py drivestat_client.pyc drivestat_server.py drive_utilization_plotter_module.py dumm.py dumm.py~ duplicate.py duplication_util.py ecron_util.py edb.py edb.pyc e_errors.py e_errors.pyc ejournal.py ejournal.pyc encp_admin.py encp.py encp.py.000 encp.pyc encp.py.with_pipe encp_rate_multi_plotter_module.py encp_ticket.py encp_ticket.pyc encp_user2.py encp_user.py encp_wrapper.py en_eval.py en_eval.pyc enmail.py enmv_admin.py enmv.py enmv_user.py enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py ENSTORE_BIN enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.pyc enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files.py enstore_files.pyc enstore_functions2.py enstore_functions2.pyc enstore_functions3.py enstore_functions3.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_pg.py enstore_plots.py enstore_plotter_framework.py enstore_plotter_module.py enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.pyc enstore_saag.py enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_system_html.py enstore_up_down.py enstore_up_down.pyc enstore_user2.py enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py esgdb.py espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_family_analysis_plotter_module.py fileinfo.py file_utils.py find_pnfs_file.py find_pnfs_file.pyc flip_tab_acsls.sh flip_tab_das.sh from_gssensrv2 ftt_driver.py ftt.py gccensrv2 generic_alarm.py generic_client.py generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.pyc get_all_bytes_counter.py GET_BIN get_cron_title.py get_hosts.py get.py GET_TEMP get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.pyc idlemovers.py info_client.py info_client.pyc info_server.py inquisitor.ann inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor.py interface.py inventory.py journal.py label_tape.py library_manager_client.py library_manager_client.py.000 library_manager_client.pyc library_manager_db.py library_manager_db.py~ library_manager.py library_manager.pyc library_manager.py.optimization_work library_manager.py.optimization_work~ library_manager.py.with_purge lintit lm_list.py lm_list.pyc lm_tick lm_tick~ lock_2.py lock_2.py~ lock_os.py lock.py lock.py~ log_client.py log_client.pyc log_finish_event.py log_server.py log_start_event.py log_trans_fail.py log_trans_fail.pyc lsof.py lsof.py~ lustre_namespace.py lustre_namespace.py~ m2.py Makefile Makefile.000 Makefile.sav make_ingest_rates_html_page.py make_migrated_as_duplicate.py make_original_as_duplicate.py makeplot.py make_pyc_files.py manage_queue_db.py manage_queue_db.py~ manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py.000 media_changer.py.1.266 media_changer.pyc media_changer.py.IBM3584 media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client1.py monitor_client1.py~ monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py mounts_plotter_module.py movcmd_mc.py mover1.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover-nanny.pyc mover.py mover.py.0000 mover.py.000001 mover.py.02-29 mover.py.1010 mover.py.1.1000 mover.pyc mover.py.Jun_27 mover.py.sav mpq_1_2.py mpq_1_2.py~ mpq_exper.py mpq.py mpq.pyc mpq.py.sav multiple_interface.py multiple_interface.pyc net_directory.py net_driver.py net_driver.pyc npipe null_driver.py null_wrapper.py on-call.py operation.py option.py option.pyc overr.py plotter_main.py plotter.py pnfs_agent_client.py pnfs_agent_client.pyc pnfs_agent.py pnfs_backup_plot.py pnfs_backup_plotter_module.py pnfsidparser.py pnfs.py pnfs.pyc priority_selector.py priority_selector.pyc put_get_nw.py put_get_nw.py~ put_get_pipe.py put_get_pipe.py~ put_get.py put_get.py~ put_get.py.000 put_get.py.0001 put_get_udp.py put_get_udp.py~ put.py q_t.py quickquota.py quota.py quota.pyc quotas_plotter_module.py ratekeeper_client.py ratekeeper_client.pyc ratekeeper_plotter_module.py ratekeeper.py ratemeter.py rate_test.py readonly_volumes.py recent_file_listing.py Release_Notes report_volume_quotas.py restoredb.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.pyc sde_admin.py SDE.py SDE.py.sav send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py slots_usage_plotter_module.py srv.py stethoscope.py string_driver.py string_driver.pyc summary_burn_rate_plotter_module.py summary_plots.py swap_original_and_copy.py tab_flipping_nanny.py take_out_acsls.sh take_out_das.sh tapes_burn_rate_plotter_module.py tcpip1.py tcpip1.py~ tcpip.py tcpip.py~ test_interval.py time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.pyc tr_test.py tr_test.py~ ttt.py ttt.py~ udp_client.py udp_client.pyc udp_cl.py udp_common.py udp_common.pyc udp.py udp.py~ udp_server.py udp_server.pyc udp_srvr.py vdbdump.py volume_assert.py volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_family.py volume_family.pyc web_server.py weekly_summary_report.py www_server.py www_server.pyc YANKED yank.py Fixed an error handling bug that would increase each resubmission by 2. 32767b acc_daily_summary.py accounting_client.py accounting_client.pyc accounting.py accounting_query.py accounting_server.py aci.py aci.py.000 aci.pyc alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.pyc aml2_dummy.py aml2_dummy.py~ aml2_dummy.pyc aml2_log.py aml2.py aml2.py.000 aml2.py.sav analize_queue.py atomic.py atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py bs.py bs.py~ callback.py callback.pyc callback.py.old cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.pyc cl_info.py clnt.py config-filec configuration_client.py configuration_client.py.000 configuration_client.pyc configuration_server.py configuration_server.py.000 configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py dcache_monitor.py delete_at_exit.py delete_at_exit.pyc discard_copy.py discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.pyc drivestat2.py drivestat_client.py drivestat_client.pyc drivestat_server.py drive_utilization_plotter_module.py dumm.py dumm.py~ duplicate.py duplication_util.py ecron_util.py edb.py edb.pyc e_errors.py e_errors.pyc ejournal.py ejournal.pyc encp_admin.py encp.py encp.py.000 encp.pyc encp.py.with_pipe encp_rate_multi_plotter_module.py encp_ticket.py encp_ticket.pyc encp_user2.py encp_user.py encp_wrapper.py en_eval.py en_eval.pyc enmail.py enmv_admin.py enmv.py enmv_user.py enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py ENSTORE_BIN enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.pyc enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files.py enstore_files.pyc enstore_functions2.py enstore_functions2.pyc enstore_functions3.py enstore_functions3.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_pg.py enstore_plots.py enstore_plotter_framework.py enstore_plotter_module.py enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.pyc enstore_saag.py enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_system_html.py enstore_up_down.py enstore_up_down.pyc enstore_user2.py enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py esgdb.py espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_family_analysis_plotter_module.py fileinfo.py file_utils.py find_pnfs_file.py find_pnfs_file.pyc flip_tab_acsls.sh flip_tab_das.sh from_gssensrv2 ftt_driver.py ftt.py gccensrv2 generic_alarm.py generic_client.py generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.pyc get_all_bytes_counter.py GET_BIN get_cron_title.py get_hosts.py get.py GET_TEMP get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.pyc idlemovers.py info_client.py info_client.pyc info_server.py inquisitor.ann inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor.py interface.py inventory.py journal.py label_tape.py library_manager_client.py library_manager_client.py.000 library_manager_client.pyc library_manager_db.py library_manager_db.py~ library_manager.py library_manager.pyc library_manager.py.optimization_work library_manager.py.optimization_work~ library_manager.py.with_purge lintit lm_list.py lm_list.pyc lm_tick lm_tick~ lock_2.py lock_2.py~ lock_os.py lock.py lock.py~ log_client.py log_client.pyc log_finish_event.py log_server.py log_start_event.py log_trans_fail.py log_trans_fail.pyc lsof.py lsof.py~ lustre_namespace.py lustre_namespace.py~ m2.py Makefile Makefile.000 Makefile.sav make_ingest_rates_html_page.py make_migrated_as_duplicate.py make_original_as_duplicate.py makeplot.py make_pyc_files.py manage_queue_db.py manage_queue_db.py~ manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py.000 media_changer.py.1.266 media_changer.pyc media_changer.py.IBM3584 media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client1.py monitor_client1.py~ monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py mounts_plotter_module.py movcmd_mc.py mover1.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover-nanny.pyc mover.py mover.py.0000 mover.py.000001 mover.py.02-29 mover.py.1010 mover.py.1.1000 mover.pyc mover.py.Jun_27 mover.py.sav mpq_1_2.py mpq_1_2.py~ mpq_exper.py mpq.py mpq.pyc mpq.py.sav multiple_interface.py multiple_interface.pyc net_directory.py net_driver.py net_driver.pyc npipe null_driver.py null_wrapper.py on-call.py operation.py option.py option.pyc overr.py plotter_main.py plotter.py pnfs_agent_client.py pnfs_agent_client.pyc pnfs_agent.py pnfs_backup_plot.py pnfs_backup_plotter_module.py pnfsidparser.py pnfs.py pnfs.pyc priority_selector.py priority_selector.pyc put_get_nw.py put_get_nw.py~ put_get_pipe.py put_get_pipe.py~ put_get.py put_get.py~ put_get.py.000 put_get.py.0001 put_get_udp.py put_get_udp.py~ put.py q_t.py quickquota.py quota.py quota.pyc quotas_plotter_module.py ratekeeper_client.py ratekeeper_client.pyc ratekeeper_plotter_module.py ratekeeper.py ratemeter.py rate_test.py readonly_volumes.py recent_file_listing.py Release_Notes report_volume_quotas.py restoredb.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.pyc sde_admin.py SDE.py SDE.py.sav send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py slots_usage_plotter_module.py srv.py stethoscope.py string_driver.py string_driver.pyc summary_burn_rate_plotter_module.py summary_plots.py swap_original_and_copy.py tab_flipping_nanny.py take_out_acsls.sh take_out_das.sh tapes_burn_rate_plotter_module.py tcpip1.py tcpip1.py~ tcpip.py tcpip.py~ test_interval.py time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.pyc tr_test.py tr_test.py~ ttt.py ttt.py~ udp_client.py udp_client.pyc udp_cl.py udp_common.py udp_common.pyc udp.py udp.py~ udp_server.py udp_server.pyc udp_srvr.py vdbdump.py volume_assert.py volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_family.py volume_family.pyc web_server.py weekly_summary_report.py www_server.py www_server.pyc YANKED yank.py Split get_clerk_info() functionality into get_file_clerk_info() and get_volume_clerk_info(). The original is now implemented using the to new functions. "Get" can use the newer versions more efficently. 32767b acc_daily_summary.py accounting_client.py accounting_client.pyc accounting.py accounting_query.py accounting_server.py aci.py aci.py.000 aci.pyc alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.pyc aml2_dummy.py aml2_dummy.py~ aml2_dummy.pyc aml2_log.py aml2.py aml2.py.000 aml2.py.sav analize_queue.py atomic.py atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py bs.py bs.py~ callback.py callback.pyc callback.py.old cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.pyc cl_info.py clnt.py config-filec configuration_client.py configuration_client.py.000 configuration_client.pyc configuration_server.py configuration_server.py.000 configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py dcache_monitor.py delete_at_exit.py delete_at_exit.pyc discard_copy.py discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.pyc drivestat2.py drivestat_client.py drivestat_client.pyc drivestat_server.py drive_utilization_plotter_module.py dumm.py dumm.py~ duplicate.py duplication_util.py ecron_util.py edb.py edb.pyc e_errors.py e_errors.pyc ejournal.py ejournal.pyc encp_admin.py encp.py encp.py.000 encp.pyc encp.py.with_pipe encp_rate_multi_plotter_module.py encp_ticket.py encp_ticket.pyc encp_user2.py encp_user.py encp_wrapper.py en_eval.py en_eval.pyc enmail.py enmv_admin.py enmv.py enmv_user.py enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py ENSTORE_BIN enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.pyc enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files.py enstore_files.pyc enstore_functions2.py enstore_functions2.pyc enstore_functions3.py enstore_functions3.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_pg.py enstore_plots.py enstore_plotter_framework.py enstore_plotter_module.py enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.pyc enstore_saag.py enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_system_html.py enstore_up_down.py enstore_up_down.pyc enstore_user2.py enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py esgdb.py espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_family_analysis_plotter_module.py fileinfo.py file_utils.py find_pnfs_file.py find_pnfs_file.pyc flip_tab_acsls.sh flip_tab_das.sh from_gssensrv2 ftt_driver.py ftt.py gccensrv2 generic_alarm.py generic_client.py generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.pyc get_all_bytes_counter.py GET_BIN get_cron_title.py get_hosts.py get.py GET_TEMP get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.pyc idlemovers.py info_client.py info_client.pyc info_server.py inquisitor.ann inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor.py interface.py inventory.py journal.py label_tape.py library_manager_client.py library_manager_client.py.000 library_manager_client.pyc library_manager_db.py library_manager_db.py~ library_manager.py library_manager.pyc library_manager.py.optimization_work library_manager.py.optimization_work~ library_manager.py.with_purge lintit lm_list.py lm_list.pyc lm_tick lm_tick~ lock_2.py lock_2.py~ lock_os.py lock.py lock.py~ log_client.py log_client.pyc log_finish_event.py log_server.py log_start_event.py log_trans_fail.py log_trans_fail.pyc lsof.py lsof.py~ lustre_namespace.py lustre_namespace.py~ m2.py Makefile Makefile.000 Makefile.sav make_ingest_rates_html_page.py make_migrated_as_duplicate.py make_original_as_duplicate.py makeplot.py make_pyc_files.py manage_queue_db.py manage_queue_db.py~ manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py.000 media_changer.py.1.266 media_changer.pyc media_changer.py.IBM3584 media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client1.py monitor_client1.py~ monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py mounts_plotter_module.py movcmd_mc.py mover1.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover-nanny.pyc mover.py mover.py.0000 mover.py.000001 mover.py.02-29 mover.py.1010 mover.py.1.1000 mover.pyc mover.py.Jun_27 mover.py.sav mpq_1_2.py mpq_1_2.py~ mpq_exper.py mpq.py mpq.pyc mpq.py.sav multiple_interface.py multiple_interface.pyc net_directory.py net_driver.py net_driver.pyc npipe null_driver.py null_wrapper.py on-call.py operation.py option.py option.pyc overr.py plotter_main.py plotter.py pnfs_agent_client.py pnfs_agent_client.pyc pnfs_agent.py pnfs_backup_plot.py pnfs_backup_plotter_module.py pnfsidparser.py pnfs.py pnfs.pyc priority_selector.py priority_selector.pyc put_get_nw.py put_get_nw.py~ put_get_pipe.py put_get_pipe.py~ put_get.py put_get.py~ put_get.py.000 put_get.py.0001 put_get_udp.py put_get_udp.py~ put.py q_t.py quickquota.py quota.py quota.pyc quotas_plotter_module.py ratekeeper_client.py ratekeeper_client.pyc ratekeeper_plotter_module.py ratekeeper.py ratemeter.py rate_test.py readonly_volumes.py recent_file_listing.py Release_Notes report_volume_quotas.py restoredb.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.pyc sde_admin.py SDE.py SDE.py.sav send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py slots_usage_plotter_module.py srv.py stethoscope.py string_driver.py string_driver.pyc summary_burn_rate_plotter_module.py summary_plots.py swap_original_and_copy.py tab_flipping_nanny.py take_out_acsls.sh take_out_das.sh tapes_burn_rate_plotter_module.py tcpip1.py tcpip1.py~ tcpip.py tcpip.py~ test_interval.py time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.pyc tr_test.py tr_test.py~ ttt.py ttt.py~ udp_client.py udp_client.pyc udp_cl.py udp_common.py udp_common.pyc udp.py udp.py~ udp_server.py udp_server.pyc udp_srvr.py vdbdump.py volume_assert.py volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_family.py volume_family.pyc web_server.py weekly_summary_report.py www_server.py www_server.pyc YANKED yank.py Moved where items are removed from the ticket to subit_one_request() from handle_retries().
Fixed a bug that prevented encp from handling CRCs that did not fit in a C-style signed int.
Fixed a problem with the crc check. The values are compared in hex, but encp did not correctly handle zero padded hex strings versus non zero padded hex string. Thus the comparisons would not match despite being correct.
Fixed a recently introduced bug in encp. The change was added for "Get". The broken change prevented normal encps from updating their metadata correctly. Now it correctly handles the situation for encp and get cases.
Fixed bug(s) relating to detecting the correct enstore system to use based on the volume.
Address error handling in the new layer 2 crc check.
Added a working check for encp to compare the crc with the crc stored in layer 2 from dcache.
Added the CRC comparison with the layer 2 dcache crc. This code is left commented out, because of an issue with the seed value used to calulate the crcs. Also, did some cleanup because a newer pychecker complained about things.
Changed the way emailable alarms are handled. Previously, emailable alarms were also sent a normal alarm copy. Now only the normal alarm is sent when the error is not emailable.
1) Fixed bug when writing large files from dcache. 2) create_zero_length_files() will raise exceptions instead of quiting (the default) if an argument is specifed. (Need for get). 3) Added additional verbose messages to set_pnfs_settings(). 4) For "get" with --list, basename is applied to specified files.
Moved the signal setup calls to earlier in the code. Also, fixed a bug the prevented encp from finding the correct enstore system for volume reads.
Added some verbose output for some get debugging.
Fixed a traceback that would occur if the pnfs id specifed with dcache writing/reading did not exists. Now an error message is given. Also, another dcache related fix. When writing (with --put-cache) encp will now check to make sure the filesizes of the input file matches that of the output file. Dcache is supposed to set the output filesize before starting encp, but that may not always be happening.
Added the --bypass-filesystem-max-filesize_check option to avoid failing a transfer because the filesystem does not support large files.
Fixed a bug for encp --volume or Get when the request volume is not declared in any enstore system. Instead of tracebacking, the user will get a NOVOLUME error.
Fixed a bug that broke encp's ability to switch between the different pnfs paths on reads.
Modified the set_pnfs_settings() function to use the file family from the volume family rather than from unreliable existance of a stand alone file family element in the dictionary.
Minor change to the list of exceptions that are more likely to happen.
Fixed some problems with the formation of valid requests. Specifically dealing with known metadata that contains different filename information that what is passed to "Get" via --list.
Modification to handle an error situation when the file does not exist on --get-bfid reads. A traceback was generated that should have been caught. This type of error was found when trying to read a file file a valid i-node but the directory entry (supposidly) pointing to it was broken (too many i-nodes for valid directory entries problem of CDF).
Speeded up the create_read_request() function for volume based transfers.
Fixed a spelling error.
Finished modifying the ability to determine which enstore system a particular volume belongs to. This is used for "Get" and the encp --volume option.
Changed submition to submission. Also, made a change to submit_one_request() to change the 'method' ticket item to "read_tape_start". Retries were failing because it was still holding 'read_next'.
More "Get" fixes. Currently, files without metadata and --list is specified can be read.
Added two return statements so that the real error gets returned right away for set_pnfs_settings().
Changed some log messages. "Waiting" -> "Listening"
These changes are for get to properly determine what metadata is available.
More edits for "Get". These include more create_read_request() changes in encp.py and a go-back-to-step-one loop in get.py.
Encp will now record the crc in layer 4 of the file. All internal encp checks have been updated to handle this. Also, all internal changes to pnfs.py have been made to handle this extra piece of data.
Fixed a bug preventing encp from reading files without brands. Also, made a change to remove a pychecker warning on IRIX.
Handle the non-optional routing communication for "Get" correctly.
Fixed a problem with encp.py working with get.py correctly.
bumping version to v3_1 because of encpCut
encp.py now uses fullpath() from enstore_functions2.py. Fixes include improved usage of the newly rewritten pnfs.is_pnfs_path() function. This allows for better checking for "unix to unix" or "hsm to hsm" at the beginning of the program, but still leaves the actual existance, permission, etc. checks for later.
bumping version to v3_0 because of encpCut
Fixed more branding problems. Clamped down on valid brands. Fixed bug that returned default csc even after determining which csc it should use.
Fixed problems with branding and determining which enstore system has the requested file.
Fix bug preventing encp from reading files with thier bfid brands containing non-alphanumeric characters (notably B0MS and D0MS with zeros).
For some reason the 'inode' field was always set to 0 instead of the actual inode. The actual inode is now used.
Added 'address' to the volume clerk sub-ticket.
Handle a "Get" related problem when the filesize is not known and using python 2.0.
Changes for mylint.py to be happy.
Another change for encp.py to support "Get". This time, the 'mover' field is not removed from the ticket before a ticket is resent to the LM.
First commit of encp after starting modification for use in the "get" program for SDSS and KTEV. All pre-get functionality should continue to work as it did before.
Slight modification to the EncpInterface file consitancy check. When reading if the directory two (or more) levels up did not exist, then a cryptic ENODATA error was given to the user. Now it is "No such file or directory".
Fixed a permission bug. A number of ways could trigger it, but the original was a regular user trying to write into a directory owned by root. In this case the user got an error of ENODATA. All changes were in e_access().
Added accounting changes to record the new rates.
new fields in encp_xfer
Modified the dcache write error when the file has already been written to reflect that the file already exists. Added the new (and old) rates to the log file message in such a way that does not break the inquisitor (hopefully).
Volume assert was broken. Is fixed now.
Fixed a bug where the target (or possibly tag) does not exist, because the directory it was supposed to be in does not exist. The bug was that it should have reported the diretory as being missing but was incorrectly stating the target.
Included to modifications to error/warning messages. The log message, "unable to register bfid" has been removed. The "Paranoid CRC mismatch" message has been changed to "CRC readback mismatch".
These changes have the encp wait for the mover to send one byte on the data socket when writing. This gives encp a chance to know when the mover is done mounting and positioning the tape on writes. This gives encp a chance at getting more acurate rates.
Fixed a bug where the wrong error message was given on writes and the file does not exist. The code was trying to test a non-pnfs file as a pnfs file and was then giving the wrong error message.
Modified encp.py and pnfs.py to agree on what a pnfs directory is. Previously, encp would consider any directory named /pnfs to be a pnfs directory. The "enstore pnfs" commands were more stringent and required /pnfs to be the very first directory in the path. This current fix is based on the looser encp version. In addition to just checking the pathname, a check for a pnfs specific "hidden" file is also performed.
Modifed the final message that encp gives (with verbosity) to include all of the rates calculated (overall, transfer, network, tape/drive and disk).
Fixed some rate calculation and reporting problems/inconsistancies. There are now 5 rates: network, tape, disk, overall and transfer.
bumping version to v2_20_b because of encpCut
Cleanup of the last sys.exc_info()/tb resource leak.
Modified to avoid things like: exc, msg, tb = sys.exc_info() and instead do something like: exc, msg = sys.exc_info()[:2] Otherwise letting the traceback get set to a local variable (tb) causes a cyclic reference.
Removed a debugging print statement.
Added some timing diagnostics to the verbose equal to level 9 output. This mostly includes the time it takes to set the file metadata on writes.
Modified encp to use host_config.update_cached_config() to force encp to use updated routes on machines where/when the enstore.conf file changes.
Added a check for layer 1 and layer 4 being empty on writes from dcache. Before this check it appears the dcache occasionaly could write the same file twice creating two (nearly) identical files in enstore. Now the second attempt will fail.
On reads, encp should now try all three paths to a file in pnfs (/pnfs/xyz, /pnfs/fs/usr/xyz and /pnfs/fnal.gov/usr/xyz).
Fixed one problem with the CRC data access value. Most changes though, deal with adding the --override-ro-mount option.
Added the crc value to the data access layer output.
On a multi-file read, the tape with the last set of files would 'win' in determining the exit status of the entire encp process. This has been fixed. This did not effect single read requests, multifile reads involving a single tape or writes. If the routing socket udp_server object failed to reaquire the same port when reinitializing; the code used to error out. Now on this 'error' it will reaquire an unused port. This may cause some 'Connection timed out' errors from the mover, but that is life. Lastly, some underpinnings for printing the CRC in the data_access_layer have been included. However, this information is not used yet for anything.
bumping version to v2_20_a because of encpCut
When encp stopped using interface.py and started using option.py the switches --ephemeral and --file-family were accidentally dropped.
Encp will no longer raise alarms if non-critical server are not accessible.
Fixed an error handling bug when generating an error message. The error was a server replying with OK, but not returning everything it should with a quit message.
Include conditional import for fcntl functionality. The FCNTL module is depricated in 2.2.
Added accounting server to list of servers to check when encp first starts.
Moved the setup_signal_handling and signal_handler functions into the delete_at_exit python module. This generalizes the code for encp, ensyc and entv.
For multi-file-transfers when writing, the delayed dismount for the first (n - 1) transfers will be at least 3 minutes. The last transfer will continue to use the default or user specified value.
Cleaned up some of the direct i/o code.
Fixed a bug regarding --ecrc and the permissions needed for the original file. It should have only required readable not read/writable.
1) The message at the beginning of the program of the tags would give incorect values if the output specified was the directory of the target and not the target itself. 2) If the alarm or log servers are not running, a warning should appear, but the code should not abort the transfer. Previous to the fix it would abort.
Fixed a bug that caused the real error reported from the mover to be ignored. Also, fixed an error message spelling error.
These changes are in advance of encp v3. They are for future files to use them based on the user being an admin or user.
Fixed a bug that prevented --threaded from working with the new option.py handling of command line parsing.
Includes changes in the paranoid ECRC message. Supports e-mailable alarms. Switched over to option.py from interface.py for interpreting the command line arguments.
feeding information to accounting server
Include support for --ecrc in encp. This option (for reads) means that after the transfer is complete encp will run an ecrc check back over the file.
Handle mover timeouts better internaly. Set the error to be RESUBMITTING instead of ETIMEDOUT when no mover info is known in handle_retries().
Fix 'unique_id' traceback error on retry/resubmit.
Bug fix. There is the possibility that some elements may not exist do to errors. This fix tests for the elements existance first.
Changed comparing types of variables to type(()), type([]) or type("") to the coresponding value in modules types.
Uses the new CRC_ENCP_ERROR instead of CRC_ERROR.
Bug fixes: 1) open_control_socket() will now check the control socket after reading the inital ticket for possible socket closed by mover case. 2) handle_retries() bug fixed that could have resulted in "mover imposter" error messages. A copy of a dictionary item was returned rather than a reference to it. 3) Slow rate errors from EXfer (from the read()/write() calls hanging in the kernel to long) are now handled specially/correctly.
Fixed the os.getcwd() vulnerability. Added the hostname to the cwd output when verbosity is turned on.
Handles the socket.socket() errors now. Tickets returned by the mover are checked for consitancy (which is being very parinoid). The gids were being incorrectly listed at the beginning of the transfer.
When an error occurs on reads the output file is truncated. Also, there are now is_read and is_write functions for use in functions used in reads and writes that return true if the transfer(s) is/are reads or writes accordingly.
Minor fix for a bug that prevented writing to /dev/null under some conditions.
Small bug fix. Wrong value types were specified into some print_data_access_layer_format() calls.
Moved where the $ENSTORE_SPECIAL_LIB environmental variable is processed on reads. Code moved from submit_one_request() to create_read_requests().
Added --shortcut switch. This is used in conjunction with the --get-cache switch to bypass full file path extrapolation. This avoids linear searches of all pnfs ids for each component in the full file path. This works similar to the --get-bfid switch. If the --shortcut switch is not given then the original --get-cache functionality remains unchanged. This change has no effect on the --put-chache switch.
Includes a fix for the way the full file paths are put together. Also, there is now a --get-bfid option for encp.
use environment variable to specify LM for read requests
Fixed how the file-family-wrapper was read in at the begining for verbose output. It was always returning as "Unknown" because there was an extra set of () trying to call the integer returned as a function.
Modified to call verify_read/write_request_consistancy() after receiving callback ticket from mover.
Included the current working directory in the verbose and log output.
Modified mover_handshake to return complete tickets. This should fix the "Error processing retry of <unique-id>" errors. Also, modified the code to be more resilant against 'problem' tickets when processing them in handle_retries(). Specificaly, with the 'resubmits' key.
Encp checks to see if all critical servers are running at the begging of the program. Also, encp now handles the "KeyError: system_inhibit" traceback error.
Fixed the format of one of the new verbose output lines.
Added gid/group and uid/user_name info to log and terminal output. Modified the request verify functions incase the database contains incomplete data. On writes, the tag info is included in the verbose output.
Fixed a bug so the previous version passes my (current) set of regression tests. The bug was that mmap io needs a file opend for read and write. If the file only has read permissions encp will fall back to posix io.
These files (encp.py, interface.py, EXfer.c) allow for encp to accept which EXfer optimizations to use from the command line.
Encp now passes more paramaters to EXfer. These allow for encp to optionally select which optimizations: threaded, direct io or memory mapped io; to use. Requires Exfer.c 1.71 to work.
Reuse the address and port when reinitalizing the routing socket on retry/resubmit.
Fixed "Linix" vs. "Linux" bug for direct io. Better error handling when direct io no implemented.
Undue previous commit.
Put in hack to handle calculating write rates when very large buffer sizes are used.
Modified encp to use O_DIRECT on xfs on linix.
bumping version to v2_20 because of encpCut
Modified the code to handle the timing statistics from the threaded version of EXfer.c.
bumping version to v2_19 because of encpCut
Fixed more potential problems with errno usage.
Included a check on the input file for reads to make sure the os filesize and the pnfs filesize match.
Fixed a bug when writing to enstore from dcache. If the file only had read permissions then an unecessary test for write permissions (file and directory) would falsly fail the transfer.
Fixed a dcache interfacing problem. Encp was trying to make sure the output file had write priledges. It shouldn't even care when dcache is involed what the output files permissions are. I cannot be assumed that all system exceptions contain the attribute errno. Defaults are now in place where needed.
Moved the FSCORRUPTED test to atomic.py. This removed doing a lisdir for each write transfer.
Moved a call to os.listdir() outside of a loop. This could be a permformace hit for large directories and multi-file transfers. Added the unique id to some more log messages...
Added some log messages. These are to help trace when encp does certain things. Others are to make the unique_id more usable.
Cleaned up the code to use new functions from e_errors for checking the status fields of tickets.
Spelling fix: recieved -> received. Created the setup_signal_handling() function. Moved code from "__main__" to do so.
Added a log message that associates unique_id with filenames. Changed FNCTL.O_NONBLOCK to os.O_NONBLOCK in anticipation of using python 2.2.
bumping version to x2_18_2 because of encpCut
Fixed an internal comment.
Made changes to handle problems with 64bit OSF nodes making encp requests. These machines could handle file sizes larger than 2GB-1 within a C int type variable. When the integer is placed in the stringified dictionary it was not containing an appending L for long type. It was this missing L that caused the problemd for the library manager since it runs on a 32bit Linux node.
venor is now spelled vendor. "unable to registor bfid" error now a warning not informational. Bug fix to the internal error handling mechanism.
bumping version to x2_18_1 because of encpCut
move non enstore import functions to enstore_functions2
Change to pass mylint.py.
Three bug fixes. 1) Empty permissions left after read. 2) OS file size zero and pnfs layer 4 size correct after writes. 3) Files writing under 2) were allowed to be read.
bumping version to v2_18 because of encpCut
Fixed one exception catch where the msg.errno attribute was accessed, but there isn't always an errno attribute.
Undo a development change that should have been done for the previous commit.
After timing out waiting for the mover to call back, reinitialize the routing socket server before resending the ticket to the library manager.
Contains a number of fixes. pathconf errors are handled better. encp_interface is added to ENCP_XFER log message. Sufficent disk space checked on reads. Some changes in routing. New interface between encp.py and EXfer.c. Needs EXfer.c 1.58 or higher.
Networking fix for the d0ola cluster.
bumping version to v2_17 because of encpCut
Encp was retrying to early. Up the wait time from 30 seconds to 15 minutes (standard timeout time). This only had to do with multiple interfaces.
Handle dcache filesize setting correctly.
Error handling bug fix for filesystem_check().
bumping version to v2_16 because of encpCut
add more information in conflict alarm
Added the user (effective and real) name to the log file output.
Removed an eroneous error message.
Added username to things printed at beginning of encp with verbosity turned on. Added KeyError to list of exceptions caught when an error occurs trying to determine ownership of files.
Encp was incorrectly putting the remote filename in the fullname section of the wrapper subticket. It now uses the local filename.
Put in signal handling code to make encp.py pass pychecker leaving only one warning left. Removed a debug print statement. Added inputfile_check() to create_write_requests().
bumping version to x2_16_1 because of encpCut
Fixed little things, mostly those found by pychecker.
"volume <volume> is maked DELETED" errors shoud be "file <filename> is marked as deleted".
There was a problem if the /etc/enstore.conf file was not there. Fixed.
Added the start time, command line and version to the stdout output if verbosity turned on. Also, made a change in regard to the reporting of the final error status.
This version appears to handle antispoofing! Only needs testing.
Added the function reply_to_caller_using_interface_ip() to udp_server. Added a __del__() function to cleanUDP. encp.py uses reply_to_caller_using_interface_ip() for the routing reply.
Finally the first version that works with the new mover/encp handshake.
If all goes well the versions of these files will fix the anti-spoofing problem.
The routes were being deleted to early for the TCP encp-mover data transfer. Some packets were being sent incorrectly.
The tickets would grow in size for each file in a multi-file transfer. This no longer happens. Also, read requests are sorted by the location cookie before being sent to the library manager.
These files were change how routes are deleted. The first enstore/encp command given will setup the static routes to the server nodes. They will remain in place and will not be deleted. The encp transfers with the mover nodes will have their routes deleted.
These changes allow for encp/enstore to correctly select which ethernet interface to send outgoing packets on. (This is the version done after looking at the packets with the analyzer.)
Fixes a problem if the destination "file" is /dev/null.
This version includes the fix for the D0 permission denied error when the local file does not have write permissions on writes. The actual failure occurs when trying to set metadata without write permissions on the real file. This fix moves the setting of the file permissions to after setting the filesize (which must be the last metadata set), but done so in such a way not to cause a major failure should an error occur.
Bug fix inside of e_access(). And the first have of the bug fix for the "CMS marked as deleted bug."
Bug fix for the last access and modififed times not being updated correctly.
The last access and modificatin times are now updated for the input and output files, respectively.
The storage group has been added to ENCPs EXFER log message. This data will be used by the inquisitor for web page information.
Second attempt at the route selection problem.
e_access(): encp now uses this instead of os.access(). os.access() uses the real uid/gid and not the effective uid/gid.
Revert to previous version. There was a problem with Linux.
These changes alter the way that routes are selected and configured. This should fix the ACLS problem (hopefully).
bumping version to v2_15 because of encpCut
Fix for dcache. The output file already exists, skip the creation step.
NOACCESS/NOTALLOWED errors did not print a data access layer if they were discovered inside create_read_requests(). Now it is printed.
There was the possibility of crashing trying to unregister a bfid when there was not a bfid in the ticket. Now the KeyError is caught in addition to the IndexError.
Changed the UNKNOWN error - when the pnfs data is empy - to PNFS_ERROR. Also, added yet another fix for get_clerks_info().
Fixed two bugs that were the same problem (from copy|paste). Two () should have been [].
Added a \n to a write to stderr. If get_clerks_info() failed to obtain an error free ticket from the file clerk, it would crash. It now checks the status field.
Fix for a bug when the library manager responce ticket is not obtained.
These changes will send the deleted=yes condition for all files left in its list of unfinished transfers. There was the possibility that an error would occur after encp was done with the mover. When this happened, the file was left in an 'unknown' state, now it is marked 'yes' for deleted.
Changes made so that bfid brands are used better when reading. The most important change is that there are now timeouts in the queries to the file clerk about its brand. Otherwise, it could get stuck waiting for the wrong file clerk.
Running out of space on a local device is now handled as a non-retriable error.
Removal of debugging print lines.
Better error handling wrt get_file_size(). Fix for EXfer error message formating. Added missing rmajor and rminor values that is required for cpio_odc wrapper.
same_cookie() handles disk location cookied correctly. Better error catching/reporting with discrepencies between what is in the database and in pnfs layer 4. The responce from the library manager that it is locked/readonly/writeonly has been confirmed to be interpreted correctly.
Fixed a serious bug w.r.t reads when a non-retriable error occured during a multi-file transfer.
Lots of changes. 1) Write requests are processed in the same order as writes. They are still submited as before. 2) EncpError class. 3) Removed volmap support. 4) Other misc. changes.
A change to allow encp to work on IRIX. os.path.split() behaves differently than it does on Linux.
Lots of changes: Added a "starting transfer" message. Correctly determines which volume and file clerks to contact wrt brands. Fixed error handling in set_pnfs_settings() wrt file clerk. Better error reporting for USERERRORS on file checks. Better error reporting for atomic.open() failures.
Changes to the bfid brand code.
Fixes a bug in determining which file clerk to contact when the known_config_servers field is not present in the configuration file. (ie. if it is not there it uses the default.) Also contains some cleanup of the filesystem, wrapper and library size checks.
This version has a work around the mylint.py blessing problem. It also, has a fix for the pathconf() filesystem size check because of OSF1. This is the first blessable version with BFID branding support (untested). Not to mention the first blessable version with the handle_retries() bug fixed.
Bug fix for the problem of not checking the status the LM sends back on any resend of a ticket. Note: This version does not pass mylint.py.
Cleanup.
This version contains a few various changes. 1) Constants for verbose levels. 2) Checks on the max size for a file for a wrapper, volume and filesystem. 3) Max directory components can contain 199 instead of 200 (for the NULL). 4) All interfaces can now be listed in the enstore history page, not just the default interface. 5) The error information when an error occurs doing an atomic open is no longer thrown away. 6) First attempt at handling the bfid brands correctly.
When an error occurs trying to atomically open the output file, these changes prevent the specific detailed information about the actuall error from being thrown out.
Changed a default value.
Fixed the occurence of the "no such file or directory' error that was being printed. Finished the library max filesize check.
Fix for checking the filesystem during a read.
Fixed a problem with testing the max file size a file system allows when writing.
encp now checks the maximum file size the output file system allows. If the file size is greater, then the transfer is aborted.
This version has a few changes. The maximum file/directory component can be 199 characters long instead of 200. The field used to report what machine was used in the transfer on the encp history page now uses 'data_ip' (if present) instead of 'host'. Other general changes.