Skip to main content

The Google File System - Google's core storage platform

Google File System - large distributed log structured file system in which they throw in a lot of data. Reliable scalable storage is a core need of any application. GFS is Google's core storage platform.Google File System (GFS) is a proprietary distributed file system developed by Google for its own use. Its point is both to assure reliablity by using redundant copies and to allow individual most used data to selectively receive more resources (more dedicated hardware or/and redundant copies).GFS is optimized for Google's core data storage needs, web searching, which can generate enormous amounts of data that needs to be retained; Google File System grew out of an earlier Google effort, "BigFiles", developed by Larry Page and Sergey Brin in the early days of Google, while it was still located in Stanford. The data is stored persistently, in very large, multiple gigabyte-sized files (around 100GB) which are only extremely rarely deleted, overwritten, or shrunk; files are usually appended to or read. It is also designed and optimized to run on Google's computing clusters, the nodes of which consist of cheap, "commodity" computers, which means precautions must be taken against the high failure rate of individual nodes and the subsequent data loss. Other design decisions select for high data throughputs, even when it comes at the cost of latency.

read more digg story
Labels:

Comments

Post a Comment

Popular posts from this blog

Chips stack up in third dimension

Stacks of chips, one on top of the other, will power the next generation of superfast PCs, IBM has announced. Laying chips vertically, instead of side by side, reduces the distance data has to travel by 1,000 times, making the chips faster and more efficient. Big blue has said that it will start producing the compact silicon sandwiches in 2008. Chip manufacturer Intel has previously announced that it is also developing similar vertical chip technology. Last year, the firm unveiled a chip with 80 processing cores and capable of more than a trillion calculations per second (teraflops) that used vertical stacking technology. Other firms, such as Tru-Si, have also developed techniques for creating 3D stacked chips. High rise Today most chips are laid out side-by-side, connected by wires. The new technique involves placing chips directly on top of each other, connected by tungsten filled pipes, etched through the silicon. These "through-silicon vias" (TSV), as they are...
Post a Comment
Read more

A problem I faced with WSS 3.0 configuration

I’d like to share a typical problem recently I faced with WSS 3.0 configuration on a Windows 2003 Server. I created a virtual machine with Windows 2003 OS and named it “ myvpc ” in a hypothetical workgroup viz. “ WKGP ” (I initially planned to use it in stand-alone mode). Then I installed all available updates to the Windows Server and .Net framework 3.5 with IIS followed by Stand-alone installation of Latest WSS 3.0. The default team site and central administration site were working smooth. I even created another application using central administration. Later I changed my mind and thought that it would be better if I include this virtual machine into our company’s local intranet Domain. At this point of time I had to change the PC name from “ myvpc ” to “ SUDEV-VPC ” due to some or other reasons. Now, suddenly none of the SharePoint sites were accessible even from the “ localhost “! I opened IIS console and noticed that “SharePoint Admin” site was stopped!!! I checked the systems log...
Post a Comment
Read more

Comma separated list of values of single Database table field

Many times you need to create a comma seperated list of values in a table. Here is a line of T-SQL solution to get comma separated list of values of single field of a database table. DECLARE @commaSeparatedVal AS VARCHAR(MAX); SELECT @commaSeparatedVal = ISNULL(@commaSeparatedVal +',','') + CONVERT(VARCHAR,[SKU]) FROM PRODUCT PRINT @commaSeparatedVal
1 comment
Read more