Object storage

An integral part of any IT infrastructure is a service designed for storing large volumes of arbitrary data (documents, backups, etc.) – file service.

All files are distributed among containers – file storages. File storage namespace is common for all system users. File storage name shall comply with certain requirements, in particular:

  • Name length must be 3 to 63 characters
  • Name shall only contain lowercase Latin letters, digits, dot, and hyphen;
  • Name and IP address shall not be the same
  • Name shall not begin with dot or hyphen and shall not end with hyphen;
  • The following symbol combinations are forbidden: ‘..’, ‘-.’, ‘.-‘.

File name may not be longer than 500 characters and shall not end with a ‘/’ symbol. There is no file directory hierarchy, however, it can be emulated because file names may contain the ‘/’ symbol.

Interface structure contains a list of user’s file storages in the place of main menu. To create a new storage, click respective button. To delete a file storage, click the “x” symbol next to its name.

../_images/fs1.png

Access control

The service allows you to manage user privileges both when working with storages and files. A user, who has created a file or storage, automatically becomes its owner.

Only storage owner has full access to the storage or file. Access rights for other users shall be explicitly listed in a special access control list (Access Control List or ACL), associated with an object. There are four permission types:

  • read;
  • write;
  • ACL read;
  • ACL write.

With regard to objects, these permissions are interpreted as follows:

Access Interpretation for bucket Interpretation for object
Read Ability to get a list of objects Ability to get an object content
Write Ability to change bucket contents (add or remove objects) Ability to change an existing object
ACL read Ability to get a bucket ACL Ability to get an object ACL
ACL write Ability to change a bucket ACL Ability to change an object ACL

Additional rules:

  • the service API allows retrieving a list of only those storages, which are owned by the user sending the request;
  • File may only be deleted by file owner or storage owner
  • Storage may only be deleted by its owner.

Attention! Please be careful when changing access control lists:

  • “read ACL” right allows learning other user’s access
  • “write ACL” right allows modifying both own access rules and other users’ access rules.

In addition to individual users, rights can be Allocated to special user groups:

  • All users – all the service users, including anonymous users;
  • Authenticated users are the service users, which have successfully passed authentication procedure.
../_images/fs2.png

Metadata

Each file can have additional attributes – metadata. Some attributes are identical to standard HTTP headers and allow fine tuning of file processing by HTTP clients. In addition to standard attributes, you can create own attributes. Own attributes are shared as HTTP headers but they have x-amz- prefix. Attribute names are not case sensitive: Cache-control and cache-control correspond to the same attribute.

../_images/fs3.png

You can use the following standard attributes:

Attribute Description Example
Content-Type Document MIME type. This parameter is required for correct document display by a browser. If you specify text/plain for an html page then such page will be displayed as plain unformatted text with all tags. You are encouraged to specify document charset (UTF-8, CP1251, KOI8-R, etc.) text/html; charset=utf-8
Content-Encoding Additional document encoding. The most frequent use is document compression by some algorithm to save disk space. gzip
Content-Disposition Allows to call the “Save as” dialog window and specify a desired file name. attachment; filename=foo.bar
Cache-Control This attribute transfers directives for all caching mechanisms included in request-reply chain. For example, “no-cache” means that caching is not used at all. no-cache
Expires One more attribute related to caching. Specifies date and time when a saved document is considered outdated Thu, 01 Dec 1994 16:00:00 GMT
Content-MD5 The document checksum is presented in BASE64 tdQ2hlY2sgSW50ZWdyaXR5IQ==

Static web-sites

You can easily use file storage as a web-site with static content. You can specify both index page and the page that is displayed if requested document is not shared or other errors occur. You can set these parameters via CROC Cloud Platform management console and external API.

Public access shall be opened to all files (select all files and click Make public on the menu). To set web-site parameters, open web-site tab. Then enable web-site mode, set index page, and (optionally) error page.

../_images/fs4.png

Save changes and test web-site performance.

When accessing http://bucket.web-site.cloud.croc.ru, an index page http://bucket.web-site.cloud.croc.ru/index.html will be displayed, where bucket is a storage set in web-site mode. If you try to access a non-existing file in the storage, an error message will appear. If error page was not set or if the value set in the configuration refers to a non-existing file, then a standard message will be displayed, issued by the system for all users.

If a file name is not specified when accessing a folder, an index document stored in this folder will be displayed. For example, when accessing http://bucket.website.cloud.croc.ru/folder a document will appear, which is also accessible by the name http://bucket.storage.cloud.croc.ru/folder/index.html.

File Service Protocol (FSP)

CROC Cloud Platform file service is compatible with Amazon S3 via software interface and supports most of standard applications designed for Amazon S3.

User authentication is performed by means of creating a special signature for each request. If the service is used other than via the web-console, the necessary settings have to be preliminary acquired in the user profile (Settings link in the top right corner). The uploaded file contains service address, account name and password for signature creation. The downloaded file can be used as rc-file for Bourne Shell environment. Furthermore in the C2_PROJECT variable the project identifier shall be specified, which is currently used in operation.

../_images/fs5.png

This protocol is an add-on to standard HTTP. To PUT and GET requests are used. In addition to GET requests, HEAD request is also shared. A special POST request is used to easily upload data to the storage from HTML forms.

The stored data is returned in original format, while the service replies, including error messages, are displayed in XML format.

Objects are addressed by URL in /storage/file format. If a file is not specified in URL then target object is a storage. Additional parameters in URL are used to access special resources. For example, /foo/bar?acl is a request for access to access control lists of bar file located in foo storage. Request examples:

Request Description
GET / Get the list of storages owned by the user
GET /foo Get the list of files in foo storage
PUT /foo Create foo storage
GET /foo/bar Get a file named bar from foo storage
PUT /foo/bar Upload file named bar to foo storage
GET /foo?acl Get access control list for foo storage
PUT /foo?acl Modify the access control list for foo storage
GET /foo/?website Get website configuration
PUT /foo?website Set website configuration (enable web-site)
DELETE /foo?website Delete website configuration (disable web-site)

If an object with the same name already exists in the storage when downloading the file, it will be overwritten, and ACL will be cleared. ACL should be updated with each update of the file.

No operation is shared for modifying separate ACL fields. You can read the entire ACL entirely or upload new ACL.

A special type of PUT request is shared to copy files within the storage. If target object already exists when file is being copied, then it will be overwritten, and access list will be cleared.