Forum

Update OpenEthereum to v3.2.6 on POA Sokol/Core

Hello @poa-validators,

We’re preparing POA Sokol and Core chains for Berlin hard fork planned on May 24. To do this, the chains need to have OpenEthereum v3.2.6 on validator nodes. Please, upgrade your Sokol and Core validator node to v3.2.6.

The following instruction assumes that you are running your current Parity-Ethereum v2.7.2 node using playbooks. If you are using docker-compose, please let us know here (we will provide another instruction for docker).

Instruction:

The switching from v2.7.2 to 3.2.6 requires resyncing from scratch on another server instance (because there was POSDAO backport from these versions to 3.2.6 and OE database versions are incompatible), so you need to follow the steps below:

  1. Create a separate Ubuntu 20.04 instance with root access.

  2. Install Docker Engine and Docker Compose following the original instructions Get Docker | Docker Documentation and Install Docker Compose | Docker Documentation

  3. Clone the following repo:

    git clone https://github.com/poanetwork/validator-node-dockerized
    mv validator-node-dockerized openethereum
    cd openethereum
    
  4. If you are POA Core validator, change the following lines in the docker-compose.yml:

  • --chain=poasokol
    --chain=poacore
  • - ./key:/root/data/keys/poasokol/key
    - ./key:/root/data/keys/poacore/key
  • WS_SERVER: "https://sokol-netstat.poa.network/api"
    WS_SERVER: "https://core-netstat.poa.network/api"
  1. To be a validator, you need to have a mining address and a private key for it. Name your JSON keystore file as key and put it to the openethereum directory created on the step 3. Put keystore’s password to password file.

  2. Copy .env.example to .env and configure the .env file. There are a few settings you need to define (without square brackets):

    ETHSTATS_ID=[validator_name]
    ETHSTATS_CONTACT=[contact_email]
    ETHSTATS_SECRET=[netstat_secret_key]
    EXT_IP=[server_external_ip]
    ACCOUNT=[0x_your_mining_address]
    

    Here:

    • ETHSTATS_ID - The displayed name of your validator in Sokol Netstat or Core Netstat.
    • ETHSTATS_CONTACT - Validator’s contact (e.g., e-mail).
    • ETHSTATS_SECRET - Secret key to connect to Netstat (should be provided by POA team, please, request it if you don’t have one).
    • EXT_IP - External IP of the current server.
    • ACCOUNT - Your mining address (with leading 0x).
  3. Edit docker-compose.yml file with your favorite text editor to remove the line

    --engine-signer="$ACCOUNT"
    

    This will temporarily set you new node as a non-validator.

  4. Start your node and wait for it to be fully synced:

    docker-compose up -d
    
  5. Once the new node is fully synced, you will need to stop your old node on an old instance (playbook):

    sudo systemctl stop poa-parity
    sudo systemctl stop poa-netstats
    
  6. Once the old instance is stopped, go to your new instance and edit docker-compose.yml file to activate a validator role - add the following line to the command section of the file:

    --engine-signer="$ACCOUNT"
    
  7. Restart your new node using the following commands:

    docker-compose down
    docker-compose up -d
    

    Now, the new node should produce blocks.

Please, report here as soon as you go through the above steps.

Thank you!

3 Likes

@poa-validators This is a reminder :arrow_up:

Instructions say they assume playbook, but first step is to install docker. Is this moving us to docker?

Best, MM

1 Like

Following up on this, it seems that there are some cross-platform instructions here… @varasev is there a set of upgrade steps fror conventional VM deployment, along with docker? Thanks in advance!

Instructions say they assume playbook, but first step is to install docker. Is this moving us to docker?

Yes, it assumes that on the new VM instance you will use docker-compose instead of the old playbooks. The old VM instance should be removed after you fully sync your new docker node.

In the future, with docker-compose you will be able to upgrade a node easier than with playbook. We’ve been using the same approach on xDai for a long time.

is there a set of upgrade steps fror conventional VM deployment, along with docker?

Do you mean using (migrating to) docker-compose on the same VM where the current playbook runs?

The current steps assume that you will create a new clear VM instance, install docker-compose on it, and then fully sync your new node. Once the new node is fully synced, you should turn off (and remove) the old VM and activate mining mode on the new one (see steps 10 and 11).

1 Like

Just saw this upgrade and will perform it by tonight!

2 Likes

I just finish upgrading my core node, sokol has been up for a few days.

2 Likes

Hello @varasev, trying to understand best practices to start and stop services inside the Dockerized Validator node. I’ve synced instance as outline in steps above, then shut down my VM to create backup before enabling the --engine-signer="$ACCOUNT" . I’ve brought node back up and it is processing current blocks and attempting to sync ancient blocks, great! For some reason the Netstats client is not running and is not automatically restarting. It was running before I shut down to create backup.

What are the commands we should use to start and stop the OpenEthereum and Netstats Docker instances? Would like to understand this before replacing current node. Also, with the old Deployment Playbooks we are able to configure certain nodes for certain use cases. Where in the configuration on our main Docker Node (The main Virtual Machine) are these configuration settings stored?

The first instance I’d like to fully activate is an extra Bootnode on POA Core labeled Bootnode Chicago D. This would remain running without the --engine-signer="$ACCOUNT" or mining key. I’ll keep trying to get the Netstats client running again. Thanks for all of your help and thanks in advance for your response!

Hello @1proof

For some reason the Netstats client is not running and is not automatically restarting. It was running before I shut down to create backup.

You can view Netstat’s logs using the following command (from the directory where the docker-compose.yml file is located):

docker-compose logs --tail 100 agent

Please, post here what you see

What are the commands we should use to start and stop the OpenEthereum and Netstats Docker instances?

Stop:

docker-compose down

Start:

docker-compose up -d

View node’s logs:

docker-compose logs --tail 100 -f openethereum

we are able to configure certain nodes for certain use cases. Where in the configuration on our main Docker Node (The main Virtual Machine) are these configuration settings stored?

The config options for the node are entirely in your docker-compose.yml file. For example: validator-node-dockerized/docker-compose.yml at d367f7654befef53dbab7b0697895ecf1b8cd496 · poanetwork/validator-node-dockerized · GitHub - these are CLI options for openethereum binary. They are discribed in Configuring OpenEthereum · OpenEthereum Documentation

So, if you need to change something in the config in future, you will need to change it in the docker-compose.yml file and restart your node (see the down/up commands above)

Hi @varasev, I have tried spinning a node several times following the instructions above and every time I am getting these errors:

docker-compose logs --tail 10
Attaching to ethstats, openethereum
ethstats | [eth] =✘= Web3 connection attempt #1 failed
ethstats | [eth] =✘= Trying again in 1000 ms
ethstats | [eth] =✘= Web3 connection attempt #2 failed
ethstats | [eth] =✘= Trying again in 1500 ms
ethstats | [eth] =✘= Web3 connection attempt #3 failed
ethstats | [eth] =✘= Trying again in 2000 ms
ethstats | [eth] =✘= Web3 connection attempt #4 failed
ethstats | [eth] =✘= Trying again in 2500 ms
ethstats | [eth] =✘= Web3 connection attempt #5 failed
ethstats | [eth] =✘= Trying again in 3000 ms
openethereum | Invalid host given with --nat extip:[X.X.X.X]
openethereum | Error: 1
openethereum | Invalid host given with --nat extip:[X.X.X.X]
openethereum | Error: 1
openethereum | Invalid host given with --nat extip:[X.X.X.X]
openethereum | Error: 1
openethereum | Invalid host given with --nat extip:[X.X.X.X]
openethereum | Error: 1
openethereum | Invalid host given with --nat extip:[X.X.X.X]
openethereum | Error: 1

X.X.X.X – current external IP of my node server

sudo ufw status
30303                      ALLOW IN    Anywhere
8545                       ALLOW IN    Anywhere
8546                       ALLOW IN    Anywhere

I am not encountering any errors during setup. Are there any other ports or specific port configs need to be added? Or is it something else?

I’m working on notes to help simplify the setup process that may help. In the mean time:

Are you able to determine the public IP Address of your new Validator node instance? Your error should be resolved if you edit the .env file in the openethereum directory for your new Validator node, and change the EXT_IP variable to match the public IP Address of your new Validator node:

EXT_IP=“xxx.xxx.xxx.xxx” ← replace with your new Validator Node Public IP Address.

Once you’ve updated your .env file with public Validator node IP Address, you can restart your Validator node with following steps (working on simpler steps):

*docker-compose down*

After stop completes, bring your OpenEthereum Docker instance back up with:
docker-compose up -d

Above SHOULD resolve your issues. Still working on notes to help simplify and standardize steps. Hope this helps!

1 Like

@alexem You don’t need to open 8545 and 8546 since Netstats agent connects to the rpc internally through docker network. Did you try to temporarily off your firewall and launch the node?

Hello @poa-validators,

We have a plan for upgrading Kovan and xDai chains to Berlin - 10th and 17th of May correspondingly. We’d like to do Berlin upgrade on Sokol and Core as well. Please, upgrade your nodes following the instruction above. Once the majority of the validators does this, we will define a block number of transition to Berlin for Sokol and Core (and update spec.json file accordingly). Thank you

2 Likes

Hi @varasev , has the OpenEthereum version increased to 3.2.5 from 3.2.2? Is there an update path for this from 3.2.2 or should be start over following steps? Thanks in advance!

@1proof I just upgraded my core and sokol nodes from 3.2.2 to 3.2.5. It was pretty simple.

backup your current docker-compose.yml (not strictly necessary but just in case)
$ cp docker-compose.yml docker-compose.ymlBACKUP

Optional: if you want to see what’s changed. Fetch changes from the repo
$ git fetch

Optional: Show changes with
$ git status

stash your changes in git, this saves your changes in git
$ git stash

pull the new changes
$ git pull

now you can see what changed in the docker-compose.yml like this
$ diff docker-compose.yml docker-compose.ymlBACKUP

apply your stash which will put any changes you made previously in docker-compose.yml back
$ git stash apply

Stop the running version
$ docker-compose down

Restart with new version, docker will download the new binary and relaunch. Mine didn’t have to resync blocks. It just started right back up.
$ docker-compose up -d

@1proof Yes, to upgrade from 3.2.2 to 3.2.5 just change the line

image: openethereum/openethereum:v3.2.2-rc.1

to

image: openethereum/openethereum:v3.2.5

in your docker-compose.yml file and then restart the node with

docker-compose down
docker-compose up -d
1 Like

Both of my nodes (Sokol and Core) have been updated.

2 Likes

UPD: figured that out. It was a user error. I have entered values in .env file inside square brackets… There should be no square brackets after =, just raw values.

2 Likes

Hi @varasev, does this log look good to you? In particular LI:#0 (block 0) error and RPC: 0 conn

ethstats | [eth] ==> Got same block: 0
ethstats | [eth] =!= Blocks are different… updating block
openethereum | 2021-05-10 06:24:09 UTC Syncing snapshot 180/1245 LI:#0 7/25 peers 1 KiB chain 0 bytes queue RPC: 0 conn, 3 req/s, 50 µs
openethereum | 2021-05-10 06:24:10 UTC Not preparing block; cannot sign.
openethereum | 2021-05-10 06:24:13 UTC Syncing snapshot 181/1245 LI:#0 7/25 peers 1 KiB chain 0 bytes queue RPC: 0 conn, 3 req/s, 53 µs

My node is still catching up with sign disabled.

Yes, such logs mean your node is syncing

1 Like