[prev] 102 [next]

Hadoop DFS (cont)

Namenodes ...
  • hold file-system meta-data (directory structure, file info)
    • e.g. file info: (filename, block#, #replicas, nodes)
    • e.g. (/data/a, 1, 2, {1,3}),  (/data/a, 2, 2, {4,5}),  (/data/a, 3, 2, {3,5})
  • provides file open/close/rename operations to clients
  • determine replication and mapping of data blocks to DataNodes
  • select Datanodes to serve client requests for efficient access
    • e.g. node in local rack > node in other rack > remote node
Namenode knows file ok if all relevant Datanodes sent Bockreport
  • if not ok, replicate blocks on other Datanodes & update meta-data