Fair Data Protocol Storage

FDP Storage is a serverless web3 filesystem for organizing users' personal data implemented in Typescript.

Such data is stored using certain structures that allow the data created in one dApp to be interpreted in another dApp. The current implementation allows to create and manage pods (similar to disks in file systems), directories, and files.

The library requires the API endpoint of a Bee node to interact with the data. If you plan to do write operations, you will need to specify postage batch id. To run a local test node trying out the functionalities, you can use FDP Play.

The FDP Storage user account is a wallet based on the BIP-44 mnemonic phrase from which one can create a portable account allowing retrieving the wallet from anywhere by providing a username and a password to the library.

The library can work in the browser, in Node.js and in mobile applications using React Native. There is an implementation of Personal Storage in Golang: https://github.com/fairDataSociety/fairOS-dfs

Project development plans and details of how each of the parts works can be found in FIPs. In this repository, you can create your proposal, which will be considered and taken into account in further development.

Warning: This project is in beta state. There might (and most probably will) be changes in the future to its API and working. Also, no guarantees can be made about its stability, efficiency, and security at this stage.

Table of Contents

• Install
  • npm
  • yarn
  • Use in Node.js
  • Use in a browser using a script tag
  • Use in React Native
• Usage
• Contribute
  • Setup
  • Test
  • Compile code
• License

Install

npm

npm install @fairdatasociety/fdp-storage

yarn

yarn add @fairdatasociety/fdp-storage

Use in Node.js and browser

We require Node.js's version of at least 16.x

const FDP = require('@fairdatasociety/fdp-storage');

Use in a browser using a script tag

Loading this module through a script tag will make the fdp object available in the global namespace.

<script src="https://unpkg.com/@fairdatasociety/fdp-storage/dist/index.browser.min.js"></script>

Use in React Native

FDP Storage is ready to work with React Native. But a few shims need to be added to the initialization script of your project to make the library components work.

import 'react-native-url-polyfill/auto' // for bee-js. URL polyfill
import 'react-native-get-random-values' // for ethers.js cryptography
import '@ethersproject/shims' // commong shims for ethers.js
import 'text-encoding' // for fdp-storage to make TextEncoding work

After creating instance of FdpStorage replace bee-js upload method to the new one because of bug #757.

const fdp = new FdpStorage('https://localhost:1633', batchId)

fdp.connection.bee.uploadData = async (batchId, data) => {
  return (await fetch(fdp.connection.bee.url + '/bytes', {
    method: 'POST',
    headers: {
      'swarm-postage-batch-id': batchId,
      'swarm-encrypt': true,
      'swarm-pin': true,
    },
    body: data
  })).json()
}

Usage

Creating FDP account

import { FdpStorage } from '@fairdatasociety/fdp-storage'

const batchId = 'GET_BATCH_ID_FROM_YOUR_NODE' // fill it with batch id from your Bee node
const fdp = new FdpStorage('http://localhost:1633', batchId)
const wallet = fdp.account.createWallet() // after creating a wallet, the user must top up its balance before registration
// Associate the created wallet with the username in the smart contract.
// This method makes the account portable.
// Seed is saved encrypted in Swarm.
await fdp.account.register('myusername', 'mypassword')

// If necessary, the account can be re-uploaded to Swarm.
await fdp.personalStorage.reuploadPortableAccount('username', 'password')

Login with FDP account

const wallet = await fdp.account.login('otherusername', 'mypassword')
console.log(wallet) // prints downloaded and decrypted wallet

Creating and using a user without interacting with the blockchain

It is not necessary to register a user in a smart contract and make his wallet portable. You can create a wallet, save the mnemonic phrase locally and import this account to interact with all the data.

// Create a wallet for interacting with data.
// It does not need to be funded.
// Operations in the blockchain will not pass through it.
const wallet = fdp.account.createWallet()

// Get mnemonic phrase of the account.
// This is the key to all data in FDP Storage.
// You need to store in a safe place.
const mnemonic = wallet.mnemonic.phrase

// to access your account, you need to import the phrase
fdp.account.setAccountFromMnemonic(mnemonic)

Creating a pod

const pod = await fdp.personalStorage.create('my-new-pod')
console.log(pods) // prints info about created pod

Getting list of pods

const pods = await fdp.personalStorage.list()
console.log(pods.getPods()) // prints list of user's pods
console.log(pods.getSharedPods()) // prints list of pods that a user has added to their account

Sharing a pod

const shareReference = await fdp.personalStorage.share('my-new-pod')
console.log(shareReference) // prints share reference of a pod

Getting information about shared pod

await fdp.personalStorage.getSharedInfo(shareReference)

Saving shared pod under user's account

await fdp.personalStorage.saveShared(shareReference)

Creating a directory

await fdp.directory.create('my-new-pod', '/my-dir')

Deleting a directory

await fdp.directory.delete('my-new-pod', '/my-dir')

Uploading data as a file into a pod

await fdp.file.uploadData('my-new-pod', '/my-dir/myfile.txt', 'Hello world!')

Deleting a file from a pod

await fdp.file.delete('my-new-pod', '/my-dir/myfile.txt')

Sharing a file from a pod

const shareReference = await fdp.file.share('my-new-pod', '/my-dir/myfile.txt')
console.log(shareReference) // prints share reference of a file

Get information about shared file

await fdp.file.getSharedInfo(shareReference)

Save shared file to a pod

await fdp.file.saveShared('my-new-pod', '/', shareReference)

Getting list of files and directories with recursion or not

// with recursion
const list = await fdp.directory.read('my-new-pod', '/', true)
// without recursion
await fdp.directory.read('my-new-pod', '/')
console.log(list) // prints list of files and directories

Downloading data from a file path

const data = await fdp.file.downloadData('my-new-pod', '/myfile.txt')
console.log(data.text()) // prints data content in text format 'Hello world!'

Deleting a pod

await fdp.personalStorage.delete('my-new-pod')

Checks whether the public key associated with the username in ENS is identical with the wallet's public key

await fdp.account.isPublicKeyEqual('username')

Migrate from v1 to v2 account

Export old wallet with mnemonic

const wallet = await fdp.account.exportWallet('oldusername', 'oldpassword', {
  mnemonic: 'one two three one two three one two three one two three'
})

or with address

const wallet = await fdp.account.exportWallet('oldusername', 'oldpassword', {
  address: '0x...'
})

// ask user to top up his account, then can be started the migration process
await fdp.account.migrate('oldusername', 'oldpassword', {
  mnemonic: wallet.mnemonic.phrase
})

Documentation

You can generate API docs locally with:

npm run docs

The generated docs can be viewed in browser by opening ./docs/index.html

Contribute

There are some ways you can make this module better:

• Consult our open issues and take on one of them
• Help our tests reach 100% coverage!

Setup

Install project dependencies with

npm ci

Test

The tests run in both context: Jest and Puppeteer.

To run the integration tests, you need to use our fdp-play project.

With specific system environment variables you can alter the behaviour of the tests.

• BEE_API_URL - API URL of Bee client
• BEE_DEBUG_API_URL - Debug API URL of Bee client
• BEE_BATCH_ID - Batch ID for data uploading
• FAIROS_API_URL - FairOS API URL

There are browser tests by Puppeteer, which also provide integrity testing.

npm run test:browser

The test HTML file which Puppeteer uses is the test/testpage/testpage.html. To open and manually test FDP with developer console, it is necessary to build the library first with npm run compile:browser (running the browser tests npm run test:browser also builds the library).

Compile code

In order to compile NodeJS code run

npm run compile:node

or for Browsers

npm run compile:browser

Maintainers

• nugaon
• IgorShadurin

License

BSD-3-Clause