User Tools

Site Tools


brainstorms:research_computing_storage_infrastructure_2012

This is an old revision of the document!


Research computing storage infrastructure, 2012

Brainstorming the current status and future needs of the ILRI research computing storage infrastructure.

Current situation

  • HPC (June, 2011)
    • ~6TB of usable disk space, ~2.5TB in use right now
  • boran (database + VM server, January, 2012)
    • ~1.5TB usable disk space, ~20GB in use right now

Timeline

  • May 18: Alan, Isaac, Etienne, and Mark meet to discuss projects and upcoming storage requirements. Notable:
    • very real possibility of a getting an Illumina MiSeq (shorter reads, but lots and lots of overlapping data)
    • Cassava genome project?
  • May 24: Alan and Isaac meet with NetApp storage representative ("GK" gkumawat@techno-associates.co.ke), facilitated by George Ogoti from ICT
    • Existing NetApp is expandable, there are various options we can explore
    • GK is going to get us a quote for the following infrastructure
      • RAID-DP (NetApp's version of RAID6, 2 disk failure)
      • Site redundancy (storage syncs nightly via fiber to ICRAF)
      • ILRI site will have two controllers for high availability
      • ICRAF site will have one controller (less critical storage)
      • Capacity 12TB or 24TB (with usable space roughly half of each figure)

Proposed NetApp architecture

Proposed architecture assuming we expand ICT's existing NetApp rack with extra controllers and storage.

Key points:

  • Raw storage is sliced in several chunks and shared appropriately
  • NetApp exports CIFS shares to corporate clients and servers (users authenticate with Active Directory credentials)
  • NetApp exports iSCSI block devices to Linux servers in order to allow them to manage their own storage/access/users directly in the OS

Alternatives

brainstorms/research_computing_storage_infrastructure_2012.1337961419.txt.gz · Last modified: 2012/05/25 15:56 by aorth