Object storage

An integral part of any IT infrastructure is a service designed for storing large volumes of arbitrary data (documents, backups, etc.) – file service.

All files are distributed among containers – file storages. File storage namespace is common for all system users. File storage name shall comply with certain requirements, in particular:

  • name length must be 3 to 63 characters;
  • name shall only contain lowercase Latin letters, digits, dot, and hyphen;
  • name and IP address shall not be the same;
  • name shall not begin with dot or hyphen and shall not end with hyphen;
  • the following symbol combinations are forbidden: ‘..’, ‘-.’, ‘.-‘.

File name may not be longer than 500 characters and shall not end with a ‘/’ symbol. There is no file directory hierarchy, however, it can be emulated because file names may contain the ‘/’ symbol.

Interface structure contains a list of user’s file storages in the place of main menu. To create a new storage, click respective button. To delete a file storage, click the “x” symbol next to its name.

../_images/fs1.png

Access control

Сервис предоставляет возможность контролировать полномочия пользователей как при работе с бакетами, так и при работе с объектами. Пользователь, создавший объект или бакет, автоматически становится его владельцем.

Полный доступ к объекту или бакету есть только у владельца. Разрешения на доступ для остальных пользователей должны быть явно перечислены в специальном списке контроля доступа (Access Control List или ACL), связанном с объектом. Существует четыре вида прав:

  • read;
  • write;
  • ACL read;
  • ACL write.

With regard to objects, these permissions are interpreted as follows:

Access Interpretation for object Interpretation for object
Read Ability to get a list of objects Ability to get an object content
Write Ability to change bucket contents (add or remove objects) Ability to change an existing object
ACL read Ability to get an object ACL Ability to get a list of objects
ACL write Ability to change an object ACL Ability to change an object ACL

Additional rules:

  • the service API allows retrieving a list of only those storages, which are owned by the user sending the request;
  • file may only be deleted by file owner or storage owner;
  • storage may only be deleted by its owner.

Attention! Please be careful when changing access control lists:

  • “read ACL” right allows learning other user’s access;
  • “write ACL” right allows modifying both own access rules and other users’ access rules.

In addition to individual users, rights can be Allocated to special user groups:

  • All users – all the service users, including anonymous users;
  • Authenticated users are the service users, which have successfully passed authentication procedure.

Metadata

Each file can have additional attributes – metadata. Some attributes are identical to standard HTTP headers and allow fine tuning of file processing by HTTP clients. In addition to standard attributes, you can create own attributes. Own attributes are shared as HTTP headers but they have x-amz- prefix. Attribute names are not case sensitive: Cache-control and cache-control correspond to the same attribute.

../_images/fs3.png

You can use the following standard attributes:

Attribute Description Example
Content-Type Document MIME type. This parameter is required for correct document display by a browser. If you specify text/plain for an html page then such page will be displayed as plain unformatted text with all tags. You are encouraged to specify document charset (UTF-8, CP1251, KOI8-R, etc.) text/html; charset=utf-8
Content-Encoding Additional document encoding. The most frequent use is document compression by some algorithm to save disk space. gzip
Content-Disposition Allows to call the “Save as” dialog window and specify a desired file name. attachment; filename=foo.bar
Cache-Control This attribute transfers directives for all caching mechanisms included in request-reply chain. For example, “no-cache” means that caching is not used at all. no-cache
Expires One more attribute related to caching. Specifies date and time when a saved document is considered outdated Thu, 01 Dec 1994 16:00:00 GMT
Content-MD5 The document checksum is presented in BASE64 tdQ2hlY2sgSW50ZWdyaXR5IQ==

Static web-sites

You can easily use file storage as a web-site with static content. You can specify both index page and the page that is displayed if requested document is not shared or other errors occur. You can set these parameters via CROC Cloud Platform management console and external API.

Public access shall be opened to all files (select all files and click Make public on the menu). To set web-site parameters, open web-site tab. Then enable web-site mode, set index page, and (optionally) error page.

../_images/fs4.png

Save changes and test web-site performance.

When accessing http://bucket.web-site.cloud.croc.ru, an index page http://bucket.web-site.cloud.croc.ru/index.html will be displayed, where bucket is a storage set in web-site mode. If you try to access a non-existing file in the storage, an error message will appear. If error page was not set or if the value set in the configuration refers to a non-existing file, then a standard message will be displayed, issued by the system for all users.

If a file name is not specified when accessing a folder, an index document stored in this folder will be displayed. For example, when accessing http://bucket.website.cloud.croc.ru/folder a document will appear, which is also accessible by the name http://bucket.storage.cloud.croc.ru/folder/index.html.

File Service Protocol (FSP)

CROC Cloud Platform file service is compatible with Amazon S3 via software interface and supports most of standard applications designed for Amazon S3.

User authentication is performed by means of creating a special signature for each request. If the service is used other than via the web-console, the necessary settings have to be preliminary acquired in the user profile (Settings link in the top right corner). The uploaded file contains service address, account name and password for signature creation. The downloaded file can be used as rc-file for Bourne Shell environment. Furthermore in the C2_PROJECT variable the project identifier shall be specified, which is currently used in operation.

../_images/fs5.png

Протокол является надстройкой над стандартным протоколом HTTP. Для большинства действий применяются запросы PUT и GET. Дополнительно к запросам GET существует запрос HEAD. Существует специальный запрос POST для удобной загрузки данных в бакет из HTML-форм.

The stored data is returned in original format, while the service replies, including error messages, are displayed in XML format.

Objects are addressed by URL in /storage/file format. If a file is not specified in URL then target object is a storage. Additional parameters in URL are used to access special resources. For example, /foo/bar?acl is a request for access to access control lists of bar file located in foo storage. Request examples:

Request Description
GET / Get the list of storages owned by the user
GET /foo Ability to get a list of objects
PUT /foo Create foo storage
GET /foo/bar Get a file named bar from foo storage
PUT /foo/bar Upload file named bar to foo storage
GET /foo?acl Get access control list for foo storage
PUT /foo?acl Modify the access control list for foo storage
GET /foo/?website Get website configuration
PUT /foo?website Set website configuration (enable web-site)
DELETE /foo?website Delete website configuration (disable web-site)

If an object with the same name already exists in the storage when downloading the file, it will be overwritten, and ACL will be cleared. ACL should be updated with each update of the file.

Для копирования объектов внутри бакета существует специальная разновидность запроса PUT. Если при копировании объекта целевой объект уже существует, то он будет перезаписан и список доступа будет обнулён.