-
Notifications
You must be signed in to change notification settings - Fork 9
/
Copy pathstorage-tool.yml
587 lines (569 loc) · 22 KB
/
storage-tool.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
title: Campus file storage and transfer
description: |
Several services at Brown allow you to share and store files. This guide will let you compare the options and decide which one(s) are right for you.
icon: database
links:
- text: check our storage options
target: /services/file-storage-and-transfer
call-for-action:
text: Check Our Storage Options
href: /storage
storage_tool_header: |
This tool lets you compare the available storage options at Brown.
Answer some of these questions or select services to compare their features
and decide which of these services suit your needs.
services:
- name: "Google Drive"
description: |
Google Drive gives you space to store and share documents.
The native Google document formats allow for real-time collaboration and file history.
You can also store unconverted files of various types in your Google Drive.
It's easy to share files with members of the Brown community (including Google Groups)
and non-Brown Google accounts; files can be shared with view-only, comment, or edit access.
Google also has a really nice feature where you can scan in handwritten documents and have
them converted to text. You can access files on the web, through a mobile app, or by
installing Google Drive on your computer (which makes it act like a folder on your
computer).
- Best for: Collaboration in native Google files, easy access from anywhere, small amount of total storage, sharing with Google Groups.
- Limitations: Data transfer speeds may be very limited, Globus can provide high bandwidth data transfers.
- More info: [Documentation](https://ithelp.brown.edu/kb/48-google-drive) | [Security](https://storage.googleapis.com/gfw-touched-accounts-pdfs/google-cloud-security-and-compliance-whitepaper.pdf)
features:
- name: relative_speed
class: fast
- name: collaborative_edits
class: True
- name: shareable_link
class: True
- name: DOI_provided
class: False
- name: data_protection_snapshots
class: True
notes:
- up to 30 days
- name: data_protection_replication
class: True
- name: canvas_integration
class: False
- name: brown_network_required
class: False
- name: access_from_oscar
class: True
notes:
- accessible via Globus
- name: security
class: 2
notes:
- prevent editors from resharing
- set expiration date
- name: storage
class: 1 TB +
notes:
- Not for large storage needs
- name: max_file_size
class: 1 TB
- name: storage_warmth
class: warm
- name: campus_file_storage_non-replicated
description: |
File Services for Research provides Brown University research departments with a location in which
files can be stored, backed up, and shared with members of the Brown community using Brown ID’s and groups.
Space is allocated to each research lab or PI with an ITHelp request , security groups are required to
define access to the data. The data is protected locally via snapshots but doesn’t have
geo-redundancy (secondary copy).
- Best for: Brown faculty and staff researchers looking to store, share and protect research data.
- Accessibility: The data is accessible on Brown's campus networks (including VPN and wireless). Also accessible directly from High Performance Computing Cluster (Oscar).
- Limitations: No geo-redundancy (secondary copy)
- Rate: $50/TB/Year when storing above free/grant allocations
features:
- name: relative_speed
class: faster
- name: collaborative_edits
class: False
- name: shareable_link
class: True
notes:
- via Globus
- name: DOI_provided
class: False
- name: data_protection_snapshots
class: True
notes:
- up to 6 months
- name: data_protection_replication
class: False
- name: canvas_integration
class: False
- name: brown_network_required
class: True
notes:
- except with Globus
- name: access_from_oscar
class: True
notes:
- Multi-Protocol Share
- name: security
class: 2
notes:
- Access controlled by Brown Authentication and Security groups
- name: storage
class: 2 TB +
notes:
- 2 TB per PI
- 20 TB per grant
- Paid after that @ $50/TB/Yr. (Summed across all paid storage types)
- name: max_file_size
class: 128 TB
notes:
- 1 TB for best performance
- name: storage_warmth
class: hot
- name: campus_file_storage_replicated
description: |
File Services for Research provides Brown University research departments with a
location in which files can be stored, backed up, and shared with members of the
Brown community using Brown ID’s and groups. Space is allocated to each research
lab or PI with an ITHelp request , security groups are required to define access to
the data. The data is replicated daily to our disaster recovery site for True
geo-redundant data protection. The data is accessible on Brown's campus networks
(including VPN and wireless).
- Best for: Brown faculty and staff researchers looking to store, share and protect research data.
- Accessibility: The data is accessible on Brown's campus networks (including VPN and wireless). Also accessible directly from High Performance Computing Cluster (Oscar).
- Sharing: The data can be shared with both Brown and non-Brown collaborators via Globus.
- Rate: $100/TB/Year when storing above free/grant allocations
features:
- name: relative_speed
class: faster
- name: collaborative_edits
class: False
- name: shareable_link
class: True
notes:
- via Globus
- name: DOI_provided
class: False
- name: data_protection_snapshots
class: True
notes:
- up to 6 months
- name: data_protection_replication
class: True
- name: canvas_integration
class: False
- name: brown_network_required
class: True
notes:
- except with Globus
- name: access_from_oscar
class: True
notes:
- Multi-Protocol Share
- name: security
class: 2
notes:
- Access controlled by Brown Authentication and Security groups
- name: storage
class: 1 TB +
notes:
- 1 TB per PI
- 10 TB per grant
- Paid after that @ $100/TB/Yr. (Summed across all paid storage types)
- name: max_file_size
class: 128 TB
notes:
- 1 TB for best performance
- name: storage_warmth
class: hot
- name: Brown_Digital_Repository
description: |
The Brown Digital Repository (BDR) is a place to gather, index, store, preserve, and make
available digital assets produced via the scholarly, instructional, research, and administrative
activities at Brown.
The Brown University Library maintains the repository as a service to the Brown community; it
provides:
- A searchable index of digital objects shared by the Brown community.
- Permanent, secure storage for personal and departmental digital objects.
- Off-site backups of digital content.
- Tools for sharing and publishing digital content.
- Data curation, format migration, and preservation services.
Faculty and researchers interested in using the Brown Digital Repository as a platform for
programmatic data management, storage, and publication should contact the Library ([email protected])
for information about opportunities for research consulting and project development support.
- More info: [Documentation](https://repository.library.brown.edu/studio/about/)
features:
- name: relative_speed
class: faster
- name: collaborative_edits
class: False
- name: shareable_link
class: True
- name: DOI_provided
class: True
- name: data_protection_snapshots
class: True
notes:
- up to 60 days
- name: data_protection_replication
class: True
- name: canvas_integration
class: False
- name: brown_network_required
class: True
notes:
- except with Globus
- name: access_from_oscar
class: False
- name: security
class: 2
notes:
- Access controlled by Brown Authentication and Security groups
- name: storage
class: 1 TB
notes:
- Contact the Library for greater needs
- name: max_file_size
class: 4 GB
notes:
- 1 TB for best performance
- name: storage_warmth
class: cold
- name: LabArchives
description: |
LabArchives is a cloud-based electronic lab notebook that can be used by researchers, instructors, and
students for input and organization of laboratory data, information sharing, and collaboration, and for
saving historical versions of files. It is appropriate for use in a wide variety of laboratories, including
biological sciences, chemistry and physical sciences, and engineering, among others.
LabArchives at Brown provides unlimited storage space. The current size limit per file is 4GB.
LabArchives at Brown is not approved for storing files containing Personally Identifiable Information (PII),
Protected Health Information (PHI), or Brown Restricted Information.
- More info: [Documentation](https://library.brown.edu/info/labarchives/)
features:
- name: relative_speed
class: faster
- name: collaborative_edits
class: True
- name: shareable_link
class: True
- name: DOI_provided
class: False
- name: data_protection_snapshots
class: True
notes:
- up to 60 days
- name: data_protection_replication
class: True
- name: canvas_integration
class: True
- name: brown_network_required
class: True
- name: access_from_oscar
class: False
- name: security
class: 2
notes:
- Access controlled by Brown Authentication and Security groups
- name: storage
class: 1 TB
notes:
- Contact the Library for greater needs
- name: max_file_size
class: 4 GB
- name: storage_warmth
class: hot
- name: oscar_storage
description: |
Oscar storage, also known as Computational Data Storage, is a high-performance data storage system which is
accessible from any computer connected from Brown's campus network, or from outside the network via ssh.
What sets this option apart from the others is that it is directly connected to Brown’s primary supercomputer,
“Oscar”, making computation easier. If you don’t intend to compute your data with Brown’s supercomputer, you
may consider using Campus File Storage instead. You could also use Oscar storage for computing and then move
your results to Campus File Storage for greater accessibility, reliability, and protection.
- Synonyms: Oscar Data, HPC Storage, GPFS (historically)
- Best for: High performance storage of research data, perform computation on your data using Brown’s supercomputer
- Limitations: Not accessible on all campus networks.
- Rate: $100/TB/Year when storing above free/grant allocations
- More info: [Documentation](https://ccv.brown.edu/services/computing#high-performance-computing-(oscar)) | [Request these services](https://brown.co1.qualtrics.com/jfe/form/SV_0GtBE8kWJpmeG4B)
features:
- name: relative_speed
class: fastest
- name: collaborative_edits
class: False
- name: shareable_link
class: True
notes:
- via Globus
- name: DOI_provided
class: False
- name: data_protection_snapshots
class: True
notes:
- up to 30 days
- name: data_protection_replication
class: True
- name: canvas_integration
class: False
- name: brown_network_required
class: True
notes:
- except with Globus
- name: access_from_oscar
class: True
- name: security
class: 2
notes:
- Access controlled by Brown Authentication and Security groups
- name: storage
class: 1 TB +
notes:
- 1 TB per PI
- 10 TB per grant
- Paid after that @ $100/TB/Yr. (Summed across all paid storage types)
- name: max_file_size
class: 128 TB
notes:
- 1 TB for best performance
- name: storage_warmth
class: hot
- name: stronghold
description: |
[Stronghold](https://it.brown.edu/services/stronghold-research-environment-data-compliance) is a secure computing and storage environment that enables
Brown researchers to analyze sensitive data while complying with regulatory
or contractual requirements.
- Best for: Storing data with data usage agreements, FISMA, etc.
- Rate: $100/TB/Year when storing above free/grant allocations
- More info: [Request this service](https://www.brown.edu/cis/forms/Stronghold/shold.php)
features:
- name: relative_speed
class: faster
- name: collaborative_edits
class: False
- name: shareable_link
class: False
- name: DOI_provided
class: False
- name: data_protection_snapshots
class: True
notes:
- up to 6 weeks
- name: data_protection_replication
class: True
- name: canvas_integration
class: False
- name: brown_network_required
class: True
notes:
- except with Globus
- name: access_from_oscar
class: False
- name: security
class: 3
notes:
- Appropriate for data with strong data compliance requirements PII, HIPAA, etc
- name: storage
class: 1 TB +
notes:
- 1 TB per PI
- 10 TB per grant
- Paid after that @ $100/TB/Yr. (Summed across all paid storage types)
- name: max_file_size
class: 4 TB
notes:
- 1 TB for best performance
- name: storage_warmth
class: hot
- name: Hibernate
description: |
[Hibernate](https://docs.ccv.brown.edu/hibernate/) is a secure, reliable, research data archive solution. Hibernate is a Brown OIT archival service for the research community to migrate inactive data off active Network-attached storage (NAS) platforms onto a lower cost, long-term retention environment.
* Hibernate leverages [StarFish](https://docs.ccv.brown.edu/starfish/), an application that provides a metadata and rules-based management framework for large file systems. StarFish makes storage tiering easy: moving data, reporting, zones.
* A zone is a virtual-volume or collection of branches across different file-systems (Oscar storage, Isilion, LRS etc.). Each PI group is distributed into an individual zone. A zone provides:
* Types/sizes of files, access times
* Hot Spots (Growth, user(s)/Group(s)
* Action: Archive, Delete, Recover
features:
- name: relative_speed
class: slow
- name: collaborative_edits
class: False
- name: shareable_link
class: False
- name: DOI_provided
class: False
- name: data_protection_snapshots
class: False
- name: data_protection_replication
class: True
- name: canvas_integration
class: False
- name: brown_network_required
class: True
- name: access_from_oscar
class: False
- name: security
class: 2
notes:
- Access controlled by Brown Authentication and Security groups
- name: storage
class: 2 TB +
note: Capacity for greater storage needs, but may eventually be a small fee
- name: max_file_size
class: 4 TB
- name: storage_warmth
class: cold
questions:
- question: What is the risk classification of your data?
information: |
**[Brown Data Risk Classification](https://it.brown.edu/computing-policies/risk-classifications)** is a
system Brown has developed to classify its information assets into one of four risk-based categories
(No Risk, Level 1, Level 2, or Level 3) for the purpose of determining who is allowed
to access the information and what security precautions must be taken to protect it
against unauthorized access. It is the data and service owner’s responsibility to ensure
appropriate security measures are taken depending on the risk classification. If you
have any questions or need help, please reach out to the Information Security Group
affected_category: security
default_answer: No Risk
answers:
- answer: No Risk
category_classes:
- 0
- 1
- 2
- 3
- answer: Level 1
category_classes:
- 1
- 2
- 3
- answer: Level 2
category_classes:
- 2
- 3
- answer: Level 3
category_classes:
- 3
- question: Would you like a DOI auto-generated for your data?
information: |
A digital object identifier (DOI) is unique and persistent identifier used to identify
a physical, digital, or abstract object (e.g., journal article, research report, or data
set). The DOI remains constant over the life of the object, and as such, provides a
digital, searchable, and persistent link. The Brown University Library will help students,
faculty, and staff obtain a DOI for digital assets produced via the scholarly,
instructional, research, and administrative activities at Brown. [You can learn more
here](https://repository.library.brown.edu/studio/doi/)
affected_category: DOI_provided
default_answer: No
answers:
- answer: Yes
category_classes:
- True
- answer: No
category_classes:
- True
- False
- question: Do you need snapshots or replication of your data?
information: |
Many storage services offer the ability to take "snapshots" or backup copies of your
data. As the terms suggests, these amount to duplicate copies—frozen at a given point
in time—of your data. And if necessary, your data could be restored to a prior snapshot
or replication copy of the data.
affected_category: data_protection
default_answer: No
answers:
- answer: Yes
category_classes:
- True
- answer: No
category_classes:
- False
- True
- answer: Not sure
category_classes:
- True
- False
- question: Is integration with Canvas required?
information: |
Are you using these data for teaching purposes, and if so, do you need your students
to be able to access the data from with the Canvas learning management system?
affected_category: canvas_integration
default_answer: No
answers:
- answer: Yes
category_classes:
- True
- answer: No
category_classes:
- True
- False
- question: How much data do you have?
information: |
Most of the storage services the University offers can accommodate very large data
sets. But there are fewer options once your data grows to exceptionally large sized.
And for the exceptionally large data sets, these will also often exceed the free
allocations that Brown provides—and thus, you might need to pay a small fee for
the additional storage.
affected_category: storage
default_answer: Less than 1 TB
answers:
- answer: Less than 1 TB
category_classes:
- unlimited
- 1 TB
- 1 TB +
- 2 TB +
- answer: Greater than 1 TB
category_classes:
- unlimited
- 1 TB +
- 2 TB +
- question: Do you need access to your data from Oscar?
information:
affected_category: access_from_oscar
default_answer: No
answers:
- answer: Yes
category_classes:
- True
- partial
- answer: No
category_classes:
- True
- False
- partial
- question: Do you need to share data via a link?
information:
affected_category: shareable_link
default_answer: No
answers:
- answer: Yes
category_classes:
- True
- answer: No
category_classes:
- True
- False
- question: Do your data require "hot" or "cold" storage?
information: |
The "warmth" of a data set is essentially the frequency with which it needs to be accessed
or updated. A "hot" data set is one that you are actively working with as part of your model
fitting or analysis. A "warm" data set is one that is still sometimes accessed for reading
and writing. A "cold" data set is effectively never (or only _extremely_ rarely) accessed.
Cold storage is often referred to as "archival storage". So, cold storage solutions are most
appropriate when the analysis of the data is complete. That is, when you are done with
the data, but you are keeping them around because of either obligations from grants or
funding agencies, or just in case they are needed at some later time.
affected_category: storage_warmth
default_answer: Any
answers:
- answer: Hot
category_classes:
- hot
- answer: Warm
category_classes:
- warm
- answer: Cold
category_classes:
- cold
- answer: Any
category_classes:
- hot
- warm
- cold