Learn web3, solidity and zk

zkblock


To create this app we will be using following tools-

- [zk-block](https://github.com/heypran/zk-block) - boilerplate for zk and web3 dapps. ( NextJS, React, Typescript, Metamask)

- [web3 storage](https://web3.storage) - library for IPFS and provides an easy and scalable way to store data on IPFS

We wont be diving into UI or ReactJs part of the app. As these are outside the scope of the article.

When dealing with any storage, there are mainly two tings that we need to deal with -

- Storing data
- Retrieving data

Assuming you can handle creating a UI and user inputs in react, we can just focus on the code that will help you interact with the IPFS using `web3.storage`.

### 1. Install the web3.storage

`npm install web3.storage`

### 2. You need to get an access token

Getting an access token is free, and you can get yours at https://web3.storage/account/.

### 3. Create a storage client object

```js
const storage = new Web3Storage({ token: process.env.NEXT_PUBLIC_WEB3_KEY });
```

### 4. Store & Retrieve Files

### Storing

The Web3.Storage client's `put` method accepts an array of File objects. So, we will have to convert any data that we want to store in a `File` object.

```
const finalContent = JSON.stringify(content);
const file = new File([finalContent], await sha256(finalContent), {
      type: 'text/plain',
    });
```

Once, we have the `File` object. we can simply call `put` to store or upload the files to IPFS

```js
const cid = await storage.put([file]);
```

CID you get back from the client when uploading is the CID of the directory and not the file itself.

To link to the file itself using an IPFS URI, just add the filename to the CID, separated by a `/` like this: `ipfs://<cid>/<filename>`.

For eg, to access this blog link stored on ipfs you can use the below link -

https://ipfs.io/ipfs/bafybeicbv3pjrmkx7vy27puybjitqibzo6r5hrlc2exp6jh4kf4ijhpaqe

---

<b>Optional:</b> put also provides callback functions to show upload progress.

```js
const onRootCidReady = (cid) => {
  console.log('uploading files with cid:', cid);
};

const onStoredChunk = (size) => {
  uploaded += size;
  const pct = totalSize / uploaded;
  console.log(`Uploading... ${pct.toFixed(2)}% complete`);
};

await storage.put(files, { onRootCidReady, onStoredChunk });
```

---

### Retrieving

You can retrieve your uploaded data by -

`https://<cid>.ipfs.dweb.link/`

`https://ipfs.io/ipfs/<cid>`

or via code

```js
const web3ResponseObject = await storage.get(cid); // Web3Response object
```

The `web3ResponseObject` extends the Response object from the Web [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) with two methods that provide access to the retrieved IPFS data: `files` and `unixFsIterator()`.

The `files` method returns an array of `Web3File` objects, which represent all files contained in the content archive identified by the given CID. A `Web3File` is just like a regular web `File` object, with the addition of `path` and `cid` properties. These contain the relative path of the file within the archive and the CID of the file, respectively.

```js
const files = await web3ResponseObject.files(); // return your files
```

You can checkout the notes app in action [here](https://github.com/heypran/dotes)

**_Optional:_** Encrypting the data

Since, all the data is publicly accessible and sometimes its best to store data in encrypted form.

```js
const stringifiedContent = JSON.stringify(content);
const finalContent = CryptoJS.AES.encrypt(
  stringifiedContent,
  password
).toString();
```

and you can decrypt like below,

```js
const retrievedContent = CryptoJS.AES.decrypt(reader.result, password).toString(
  CryptoJS.enc.Utf8
);
const actualContent = JSON.parse(retrievedContent);
```

That's all!


In this article we will be creating a notes app using IPFS. IPFS is a decentralized storage.

Build a Web3 notes app in 4 steps?


## Why Storage is expensive

---

Storage is any data that is persistent between transactions. It can be accessed in future transactions.

To explain why this is the case in a blockchains, lets consider a centralized database.

On a centralized server, storage is stored on a single database ( or may be there is some redundancy). It’s predictable, so as more users use this centralized application, we can predict the amount of storage the database will need. It’s easily scalable, as there are only very few places to scale up.

On a blockchain however there are thousands of nodes and the data stored no longer exists in just one place but on tens of thousands of disks. It also has to be readily available to every node in the network if a future transaction comes along to access or change it. The overall cost of this data equates to the sum of the space it consumes across the whole network.

So, the EVM has two intentionally expensive opcodes:

#### SLOAD

Load word from storage.
Stack input - key
Min Gas cost - 100

#### SSTORE

Save word to storage.
Stack input - key & value
Min Gas cost - 100


Storage contains data that persists on chain. These are stored as part of the storage merkle tree.

Why storage is expensive?


## Abstract Contracts

---

- Contracts need to be marked as abstract when at least one of their functions is not implemented.
- Contracts may be marked as abstract even though all functions are implemented.

```js
pragma solidity >=0.4.0 <0.7.0;

abstract contract Feline {
    function utterance() public virtual returns (bytes32);
}

```

- Such abstract contracts can not be instantiated directly. This is also true, if an abstract contract itself does implement all defined functions.

```js

pragma solidity ^0.6.0;

abstract contract Feline {
    function utterance() public virtual returns (bytes32);
}

contract Cat is Feline {
    function utterance() public override returns (bytes32) { return "miaow"; }
}

```

- If a contract inherits from an abstract contract and does not implement all non-implemented functions by overriding, it needs to be marked as abstract as well.

- Abstract contracts are useful in the same way that defining methods in an interface is useful. It is a way for the designer of the abstract contract to say “any child of mine must implement this method”.

## Interfaces

---

Interfaces are similar to abstract contracts, but they cannot have any functions implemented. There are further restrictions:

- They cannot inherit from other contracts, but they can inherit from other interfaces.

- All declared functions must be external in the interface, even if they are public in the contract.

- They cannot declare a constructor.

- They cannot declare state variables.

- They cannot declare modifiers.

```js

// SPDX-License-Identifier: GPL-3.0
pragma solidity >=0.6.2 <0.9.0;

interface Token {
    enum TokenType { Fungible, NonFungible }
    struct Coin { string obverse; string reverse; }
    function transfer(address recipient, uint amount) external;
}
```

- Types defined inside interfaces and other contract-like structures can be accessed from other contracts: `Token.TokenType` or `Token.Coin`.

- All functions declared in interfaces are implicitly virtual and any functions that override them do not need the override keyword. This does not automatically mean that an overriding function can be overridden again - this is only possible if the overriding function is marked virtual.

- Interfaces can inherit from other interfaces. This has the same rules as normal inheritance.

```js

// SPDX-License-Identifier: GPL-3.0
pragma solidity >=0.6.2 <0.9.0;

interface ParentA {
    function test() external returns (uint256);
}

interface ParentB {
    function test() external returns (uint256);
}

interface SubInterface is ParentA, ParentB {
    // Must redefine test in order to assert that the parent
    // meanings are compatible.
    function test() external override(ParentA, ParentB) returns (uint256);
}
```


Abstract contracts and interfaces share some similar characterstics but they are different.

Abstract Contract vs Interfaces in Solidity


`memory` variables are temporary variables that exist only inside the calling function. The memory variables do not persist and are generally cheaper to use than storage variables.

`storage` variables are stored in contract state and are only changed by sending a transaction and spending gas.

```js

pragma solidity ^0.7.0;

contract ArrayInsideStruct {

  event CreateGame(address indexed _from, uint256 _value);

  struct Game {
    address[] users;
    uint256 gameId;
  }

  Game[] public games;

  function createGame() public {

      // initialize struct with empty new array of size 0
      Game memory game = Game(new address[](0), 0);

      games.push(game);

      // push data to the array inside struct
      games[games.length-1].users.push(msg.sender);

      CreateGame(msg.sender, games.length-1);
  }
  function getUsers(uint i) public view returns (address[]){
      return game[i].users;
  }
}
```


Using Arrays Within Solidity Structs


## Mappings in Solidity

---

Mappings are similar to hash tables that help store some data in them. They store data in the key-value pair format somewhat like objects in JavaScript or dictionaries in Python.

```
mapping(_KeyType => _ValueType) public mappingName
```

- Mappings are useful because they can hold a large number of \_KeyTypes to \_ValueTypes.
- Mappings don’t really have a length, nor do they have the concept of setting a key or a value.
- Mappings are only applicable to state variables that serve as storage reference types.
- When mappings are initialized, they contain every possible key and are mapped to values whose byte-representations are all zeros.

## Structs in Solidity

---

Solidity gives users the ability to create their own data types in the form of structures. The struct contains a collection of elements, each of which has a different data type. In most cases, it is used to represent a record. The struct keyword is used to define a structure, which creates a new data type.

Refer to the example below:

```js
Contract test {

   // Declaring a structure
   struct Human {
      string name;
      string family;
      uint age;
      bool happy;
   }
}
```

## Use `Mappings` inside `Structs` in solidity

We will now simply declare mapping inside a struct and create a function to set values to the struct.

```js
contract Crowdfund {

    // defining our Request struct
    struct Request {
        string description;
        uint256 value;
        address recipient;
        bool complete;
        uint256 approvalCount;

    }

    Request[] public requests;

    function createRequest(
        string memory description,
        uint256 value,
        address recipient
    ) public {

        Request memory newRequest = Request({
            description: description,
            value: value,
            recipient: recipient,
            complete: false,
            approvalCount: 0
        });

        // Request(description, value, recipient, false); // alternative syntax to create Request instance - NOT RECOMMENDED!

        requests.push(newRequest);
    }
}

```

To access any element of the structure, we use the dot operator, which separates the struct variable from the element we want to access. The variable of structure data type structure name is used to define it.

```js
requests[0].description;
```


Using mappings and structs in solidity can sometimes become tricky due to the nature of how solidity works.

Using Mappings and Structs in solidity


Libraries in solidity are considered stateless, and hence have the following restrictions:

- They do not have any storage (so can’t have non-constant state variables)
- They can’t hold ethers (so can’t have a fallback function)
- Doesn’t allow payable functions (since they can’t hold ethers)
- Cannot inherit nor be inherited
- Can’t be destroyed (no selfdestruct() function since version 0.4.20)

There are 2 types of libraries: **deployed & embedded**

### Deployed libraries

These have their own address, and they can be used by several other smart contracts.
If a library contain public or external functions then library needs to be deployed. The deployment of library will generate a unique address in the blockchain. This address needs to be linked with calling contract.

### Embedded libraries

These don’t have their own address and are deployed as part of the code of the smart contract that use them.
If a smart contract is consuming a library which have only internal functions, then the EVM simply embeds library into the contract. Instead of using delegate call to call a function, it simply uses JUMP statement(normal method call). There is no need to separately deploy library in this scenario.

```js

// Embedded (function is internal)
library Lib {
  function add(uint a, uint b) pure internal returns(uint) {
    return a + b;
  }
}

//  Deployed (function is public)
library Lib {
  function add(uint a, uint b) pure public returns(uint) {
    return a + b;
  }
}

contract Example {

    using Lib for uint;

    function addExample(uint _a, uint _b) public view returns (uint) {
        return _a.add(_b);
    }
}

```


A library is a piece of code that can be re-used by other smart contracts.

Libraries in solidity


## Topics

---

Topics are indexed parameters to an event.
`topic[0]` always refers to the hash of the hash of the event itself, and can have up to 3 indexed arguments, which will be reflected in the topics.

## Logs

---

EVM uses low-level primitives called logs to map them to high-level solidity construct called Event. Logs may contain different topics that are indexed arguments.

#### Example:

Consider Event:

```js
 event GroupCreated(uint indexed groupId, uint itemCount);
```

and you emit the above event inside `createGroup` function:

```js
 function createGroup() {
        emit GroupCreated(12, 23);
  }

```

This will create a low-level EVM log entry with topics

`0x123....abc` (Keccak-256 hash of GroupCreated(uint256,uint256))

`0x4238....gef` (Keccak-256 hash of groupId), value 12

You'll notice that `itemCount` will not be a topic, but it will be included in the data section of the event.

Now, in the web3 client you may watch for creation events of `groups` with `groupId` 12 Or you can also filter all past events in similar way (refer below):

```js
const event = myContract.GroupCreated({ groupId: 12 });
event.watch(function (err, result) {
  if (err) {
    console.log(err);
    return;
  }
  console.log('Found ', result);
});
```


Topics are indexed parameters to an event. EVM uses low-level primitives called logs to map them to high-level solidity construct called Event.

Relation Between Solidity Events, Topics & Logs


## Mesauring gas consumption

---

You can measure the gas used in a block of code by calculating the difference between `gasleft()` at the start and `gasleft()` at the end:

#### Example:

```js
// SPDX-License-Identifier: MIT
pragma solidity 0.8.10;

contract Gas {
    uint public i = 0;
    uint public gasUsed = 0;
    uint public testCalculation;
    uint public gasPrice = 0;

    // Using up all of the gas that you send causes your transaction to fail.
    // State changes are undone.
    // Gas spent are not refunded.
    function forever() public {
        // Here we run a loop until all of the gas are spent
        // and the transaction fails
        while (true) {
            i += 1;
        }
    }

    function checkGasUsed() public {
        uint startGas = gasleft();

        testCalculation = 25/5;

        gasUsed = startGas - gasleft(); // 22121
    }

    function checkGasUsed2(uint a) public {
        uint startGas = gasleft();

        testCalculation = a/5;
        gasPrice = tx.gasprice;
        gasUsed = startGas - gasleft(); // 27315
    }

}
```


Gas is one of most important consideration when writing smart contract. Solidity provides some built in helper funciton to measure gas.

Measure Gas Used in Solidity


Functions can be declared as

| Visibility | Description                                                        |
| ---------- | ------------------------------------------------------------------ |
| `public`   | Any contract and account can call                                  |
| `private`  | Accessible only inside the contract that defines the function      |
| `internal` | Accessible only inside contract that inherits an internal function |
| `external` | Accessible only other contracts and accounts can call              |

State variables can be declared as public, private, or internal but not external.

```js


// SPDX-License-Identifier: MIT
pragma solidity ^0.8.15;

contract ViewAndPure {
    uint public gamePoints = 0;


    // No modification or reading from the state
    function checkRewards(uint multiplier, uint points) public pure returns (uint) {
        return multiplier * points;
    }

    // No modification but can read from the state
    function addGamePoints(uint newScore) public view returns (uint) {
        return gamePoints + newScore;
    }

}

```

> If you call `view` or `pure` functions externally, you do not pay a gas fee.

[Read more about storage and memory here](./arrays-within-struct)


Functions and state variables have to declare whether they are accessible by other contracts or externally.

Visibility in Solidity


`view` indicates that the function will not alter the storage state in any way. But it allows you to read the state.

`pure` is even more strict, indicating that it will not even read the storage state.

A pure function is a function which given the same input, always returns the same output. But the state of the contract keeps changing as users interact with it. So if you pass a state variable as an argument to the function, since the state is changing, that function will not be pure. That's why pure functions cannot access to state.

```js


// SPDX-License-Identifier: MIT
pragma solidity ^0.8.15;

contract ViewAndPure {
    uint public gamePoints = 0;


    // No modification or reading from the state
    function checkRewards(uint multiplier, uint points) public pure returns (uint) {
        return multiplier * points;
    }

    // No modification but can read from the state
    function addGamePoints(uint newScore) public view returns (uint) {
        return gamePoints + newScore;
    }

}

```

> If you call `view` or `pure` functions externally, you do not pay a gas fee.

[Read more about storage and memory here](./arrays-within-struct)


View and pure functions can be confusing sometimes for the newbees in solidity. This post will clarify the major differences.

View and Pure Functions in Solidity


## Loop through Mapping in Solidity

---

There is no direct way to way loop through the keys or count the keys in a mapping, but there are ways to achieve this using some extra variables.

```js
pragma solidity 0.8.15;

contract LoopMap {

  address[] public userAddress;

  mapping (address => uint) public addresssItemMapping;


  // Option1: construct an array of items by looping the mapping inside soldity
  function getAllItems() public view returns(uint[] memory items) {

    // create an in-memory array to return as result
    uint[] memory allItems = new uint[](userAddress.length);

    // add all items to the in-memory array
    for (uint i =0;i<userAddress.length;++i){
        allItems[i]=addresssItemMapping[userAddress[i]];
    }

    return allItems;
  }

  // Option2: call this function from frontend in a loop
   function getSingleItem(address _user) public view returns(uint item) {
    return addresssItemMapping[_user];
  }

  // Option3: Use an indexing service to read/query the relevant data such as sub-graph
}
```

But you must keep in mind some of the limitations of iterations, like having too many steps to the point that the iteration exceeds the maximum amount of gas. The whole purpose of doing this is to read data from the blockchain. Another way this can be achived is by emitting the events and use event caching to query the data.


Looping through mapping in solidity can be tricky, considering the way solidity works and is a strictly compiled language.

How to loop through mapping in solidity?


## ERC677

---

Allow tokens to be transferred to contracts and have the contract trigger logic for how to respond to receiving the tokens within a single transaction.

To do an api call or vrf request requires a function call on a smart contract (ie on the the oracle contract or operator contract or vrf coordinator contract etc). To transfer a token also requires a function call on the token contract. So if you say wanted to do a vrf request and send the payment in link, usually that would be 2 seperate function calls on 2 seperate contracts. With erc677 you can basically do both in a single transaction, thanks to the magic of the onTokenTransfer and transferAndCall functions.

### Reference

- https://github.com/ethereum/EIPs/issues/677


What is ERC677? When to use it?


All function calls in Ethereum are identified by the first 4 bytes of the data payload, which is known as the function selector. The selector is calculated from a hash of the function name and its signature. Now, 4 bytes is not a lot of entropy, which means that there is potential for clashing between two functions: two different functions with different names may end up having the same selector. If you happen to stumble upon such a case, the Solidity compiler will be smart enough to let you know, and refuse to compile a contract with two different functions with different names that have the same 4-byte identifier.

```js
// This contract will not compile, as both functions have the same selector
contract Foo {
function collate_propagate_storage(bytes16) external { }
function burn(uint256) external { }
}
```

### Why function selectors matters?

When working with upgradeable contracts in solidity, it is perfectly possible for an implementation contract to have a function that has the same 4-byte identifier as the proxy’s upgrade function.

This could cause an admin to inadvertently upgrade a proxy to a random address while attempting to call a completely different function provided by the implementation. This issue can be solved either by appropriate tooling while developing upgradeable smart contracts, or at the proxies themselves. In particular, if the proxy is set up such that the admin can only call upgrade management functions, and all other users can only call functions of the implementation contract, clashes are not possible. This is one of main reason Transparent Proxy patterns were recommended.


Understanding how function selectors work can help prevent unwanted bugs in solidity.

Function Selectors in Solidity


```js

function compareLengthAndHash(string a, string b) internal returns (bool) {
    if(bytes(a).length != bytes(b).length) {
        return false;
    } else {
        return keccak256(a) == keccak256(b);
    }
}

```


Compare Strings In Solidity


`internal` is the default visibility level for state variables.

`private` state variables are like internal ones but they are not visible in derived contracts.

A `private` or `internal` function can only be called by the contract itself.

### How does the EVM enforce this?

On the EVM level, there are no function calls. Rather, solidity generates a dispatcher for each contract, and sending a transaction just activates said dispatcher. On a theoretical level, making a function private is just a matter of not putting it in the dispatcher.


Private functions are not callable from outside the contract.