---
title: S3 Protocol
weight: 3
menu:
   main:
      parent: "Client Interfaces"
summary: Ozone supports Amazon's Simple Storage Service (S3) protocol. In fact, You can use S3 clients and S3 SDK based applications without any modifications with Ozone.
---

<!---
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->


Ozone provides S3 compatible REST interface to use the object store data with any S3 compatible tools.

S3 buckets are stored under the `/s3v` volume.

## Getting started

S3 Gateway is a separated component which provides the S3 compatible APIs. It should be started additional to the regular Ozone components.

You can start a docker based cluster, including the S3 gateway from the release package.

Go to the `compose/ozone` directory, and start the server:

```bash
docker-compose up -d --scale datanode=3
```

You can access the S3 gateway at `http://localhost:9878`

## URL Schema

Ozone S3 gateway supports both the virtual-host-style URL s3 bucket addresses (eg. http://bucketname.host:9878) and the path-style addresses (eg. http://host:9878/bucketname)

By default it uses the path-style addressing. To use virtual host style URLs set your main domain name in your `ozone-site.xml`:

```xml
<property>
   <name>ozone.s3g.domain.name</name>
   <value>s3g.internal</value>
</property>
```

## Bucket browser

Buckets could be browsed from the browser by adding `?browser=true` to the bucket URL.

For example the content of the 'testbucket' could be checked from the browser using the URL http://localhost:9878/testbucket?browser=true


## Implemented REST endpoints

Operations on S3Gateway service:

Endpoint    | Status      |
------------|-------------|
GET service | implemented |

Operations on Bucket:

Endpoint                            | Status      | Notes
------------------------------------|-------------|---------------
GET Bucket (List Objects) Version 2 | implemented |
HEAD Bucket                         | implemented |
DELETE Bucket                       | implemented |
PUT Bucket (Create bucket)          | implemented |
Delete Multiple Objects (POST)      | implemented |

Operation on Objects:

Endpoint                            | Status          | Notes
------------------------------------|-----------------|---------------
PUT Object                          | implemented     |
GET Object                          | implemented     |
Multipart Upload                    | implemented     | Except the listing of the current MultiPartUploads.
DELETE Object                       | implemented     |
HEAD Object                         | implemented     |


## Security

If security is not enabled, you can *use* **any** AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY

If security is enabled, you can get the key and the secret with the `ozone s3 getsecret` command (*kerberos based authentication is required).

```bash
kinit -kt /etc/security/keytabs/testuser.keytab testuser/scm@EXAMPLE.COM
ozone s3 getsecret
awsAccessKey=testuser/scm@EXAMPLE.COM
awsSecret=c261b6ecabf7d37d5f9ded654b1c724adac9bd9f13e247a235e567e8296d2999
```

Now, you can use the key and the secret to access the S3 endpoint:

```bash
export AWS_ACCESS_KEY_ID=testuser/scm@EXAMPLE.COM
export AWS_SECRET_ACCESS_KEY=c261b6ecabf7d37d5f9ded654b1c724adac9bd9f13e247a235e567e8296d2999
aws s3api --endpoint http://localhost:9878 create-bucket --bucket bucket1
```

To invalidate/revoke the secret, use `ozone s3 revokesecret` command. Parameter '-y' can be appended to skip the interactive confirmation.

```bash
ozone s3 revokesecret
Enter 'y' to confirm S3 secret revocation for 'testuser/scm@EXAMPLE.COM': y
S3 secret revoked.
```

Ozone Manager administrators can run `ozone s3 getsecret` and `ozone s3 revokesecret` command with `-u` parameter to specify another users.

```bash
# Obtained Kerberos TGT for testuser/scm@EXAMPLE.COM with kinit,
# testuser/scm@EXAMPLE.COM is an OM admin.
ozone s3 getsecret -u om/om@EXAMPLE.COM
awsAccessKey=om/om@EXAMPLE.COM
awsSecret=1e9379d0424cce6669b1a501ff14834e46dee004ee868b41a313b49eabcfb68f

ozone s3 revokesecret -u om/om@EXAMPLE.COM -y
S3 secret revoked.
```

## Expose any volume

Ozone has one more element in the name-space hierarchy compared to S3: the volumes. By default, all the buckets of the `/s3v` volume can be accessed with S3 interface but only the (Ozone) buckets of the `/s3v` volumes are exposed.

To make any other buckets available with the S3 interface a "symbolic linked" bucket can be created:

```bash
ozone sh volume create /s3v
ozone sh volume create /vol1

ozone sh bucket create /vol1/bucket1
ozone sh bucket link /vol1/bucket1 /s3v/common-bucket
```

This example expose the `/vol1/bucket1` Ozone bucket as an S3 compatible `common-bucket` via the S3 interface.

(Note: the implementation details of the bucket-linking feature can be found in the [design doc]({{< ref "design/volume-management.md">}}))

## Clients

### AWS Cli

`aws` CLI could be used by specifying the custom REST endpoint.

```bash
aws s3api --endpoint http://localhost:9878 create-bucket --bucket buckettest
```

Or

```bash
aws s3 ls --endpoint http://localhost:9878 s3://buckettest
```

### S3 Fuse driver (goofys)

[Goofys](https://github.com/kahing/goofys) is a S3 FUSE driver. As Ozone S3 gateway is AWS S3 compatible, it can be used to mount any Ozone buckets as an OS level mounted filesystem.

```bash
goofys --endpoint http://localhost:9878 bucket1 /mount/bucket1
```
