Sokol Testnet - On Boarding Procedures

POA Core Legal
#1

Motivation:: a lot has been said, we can focus discussion in this one post about what the testnet ought to look like. I don’t know if this is the best sub; however, I think we’ve sort of co-opt’ed “Legal” for Governance related matters.

I am of the opinion that any interested persons should be allowed full and complete access to the Testnet in all forms. No requirements or restrictions should be in place.

We are a part of a permissioned public Blockchain - public being key. Some of the goals of a testnet: (in my opinion)

  • Complete inclusion to anyone without restriction - both to use for their dapps as well as to run nodes
  • Try out new constructs (in our case - this includes governance in addition to software based ideas.)
  • Education/training of node operators
  • See how the network behaves under duress

This last bit needs unpacking. Personally - I don’t like the idea of a ‘clean’ or pristine testnet (I love sewer rats). Right now, the testnet is basically a mirror of the current mainnet (core) with regards to nodes that are highly maintained - it is a thing of beauty at present. Sure we have regular unit testing and stress testing that occurs; however, I would love to see a testnet that jumps to a hundred+ validating nodes.

Opening up the testnet would result in, at least in the short term, an explosion of validating nodes…followed by a collapse within two/three months time. Imagine how valuable that would be.

Sure Parity’s Aura can handle faulty nodes; however, we understand this in a purely theoretical manner for the most part. We don’t know what edge cases we are missing. Look at Bitcoin’s testnetS - what number are we on - I forget? These had to be killed and re-formed due to unforeseeable gaming of - not like Ethereum’s testnets with parameters tweaks by different teams wanting different enviros to build within.

How would the network respond to unresponsive nodes? Again, we theoretically understand this - we would ‘remove’ bad nodes. What exactly would ‘remove bad nodes’ actually look like. There would be unresponsive node operators but also the potential of nodes that are simply faulty. I think many are under the impression that a faulty node is due to laziness of the human operator but what if there were a software issue that caused a number of machines to go offline at once.

How would the network really respond and deal with this potential? I think this will be a lot harder than anyone understands, and will result in a lot of actual work. I also am sensitive and aware that an explosion of nodes would stress other systems, such as support infrastructure. Perhaps we might need to have validators ‘pay’ for these support systems/persons in order to ensure uptime - or something.

Again - I’m just dreaming/thinking aloud. But an open testnet, one that allows more than simply running of a bootnode would help to better understand all the interconnected systems - ones that we are taking for granted at present.

3 Likes
#2

Jeff, when you say “We are a part of a permissioned public Blockchain - public being key. Some of the goals of a testnet: (in my opinion)…” do you refer to the access and use of the network or the validator participation. If the former, I believe the network is already open to the public use. If the latter then it would be wise to define the meaning and the purpose of the “validator” first. Second step would be good to understand what goals are you trying to achieve. Third is what metric tools do you have in order to measure the result. And the last, but not least is what fail-safe mechanisms and backup plans do you have in place.

1 Like
#3

Sure - I realize that anyone can deploy their dapp on the network, we give them coins to test already via the faucet.

I’m interested in making complete participation within the testnet automatic (ideally). If you would like to be a validator - great you’re one without the need to ask permission.

Reading through Aura and trying to better understand how failure modes are handled - that gets you just so far. Really pushing the limits of the network and the softwares would help understand exactly how finality would be affected.

So the goals of a testnet, see how it behaves in the wild. A lot can happen, things that nobody thought of nor built for. I dare say, 90% of Blockchain programming is building fixes for unintended gaming. Initially, I thought this to be a waste of resources - I now understand that this is just part and parcel of the human condition - and we must expend the energies.

But forget about the technical side… How would governance and internal systems handle extreme stress. How would our support infrastructure respond? Maybe if we had 100+ potential validators - it would be the push needed to take a hard look at our training and HOWTO materials.

A testnet’s sole goal is to be run into the ground hard. Then you see what went wrong, and solve the problem. Having a testnet that has undo restrictions results in a clean, stable, and ‘perfect’ environment… and you’ll never see “it” coming ( if you hadn’t built a unit test beforehand.)

1 Like
#4

Jeff, I believe we had GitHub wiki instructions posted on what is expected from potential candidates to become Sokol and Core validators. It was clearly stated that Notary license is required for both. I can imagine there are people who took those instructions seriously and waiting for the license to apply. We currently have people on Sokol who were interested to participate on the testnet but were waiting for Notary license to be approved. I believe we should have the same set of requirements for everyone. More experienced Sokol validators are considered for Core ahead of less experienced and newer members. Sokol is almost identical to Core in terms of functionality and governance model. We test our governance model there first before we make changes to Core. If testing governance system is the intent for Sokol then we should have the same set of on-boarding procedures for both Sokol and Core. I believe If we have a different set of requirements then we wont test our governance model on Sokol and will only test the functional part. Let’s say we lower the requirements on Sokol and have a rapid expansion of members who then shut the entire Sokol testnet down. Would it be a positive experience for Core? I would like to hear what you guys think. People can still participate, learn about our network, run smart contracts by running boot-nodes on Sokol and Core as well as engage in discussions on our forum. All of the above is opened for public without any permissions. You still need a permission from existing Sokol validators to become one and run a validator node. By saying you support complete inclusion to anyone without restriction, do you imply existing Sokol validators should vote in everyone? Did you mean people from other countries and people who do not intent to become Core validators are welcomed to run Sokol validator nodes and vote in/out others?

2 Likes
#5

Henry, I do believe that those pages were removed - as it was never the intent nor spirit.

That is if you are you talking about the wiki pages? The testnet was always for interested parties - without restriction.

Making restrictions on a testnet will result in a smaller testnet. A cleaner testnet. A perfect testnet. That’s not ideal in my book.

#6

Yes, the requirements were changed recently but we did not hold a discussion whether we should remove them or not.

What do you think about this?

#7

There really is a big difference between Blockchain programming and non.

I’m from the camp of highly optimized code at all costs, only recently even believing that javascript is a legit language - I can be reformed :wink:

Blockchain programming is an entirely different beast because these transactions/flat files/whatever you’d like to think of them as - actually contains stored economic energy (potentially). As such, there are a lot of attack surfaces - many that we have not discovered yet. Potential issues around the protocols as well as governance models.

The governance aspect alone is complex - look at Bitcoin, Ethereum, etc… and how those communities have changed and re-org’ed. We need to have as many potential stakeholders to see how we will change and how we can be changed. If we create a homogenous set, whereby all the nodes can be trusted to be well maintained, and have X-Y-Z certs; what happens when there’s a zig instead of a zag? Is a potentially brittle system acceptable?

So I’m of the opinion that we really beat on the testnet HARD and that means we need to allow anyone slightly interested to come aboard.

As for a discussion - I thought that is what we are doing now?

1 Like
#8

I believe if we don’t post any restrictions to become a Sokol validator and automatically accept people then a simple script would be able to create fake identities, start a validator node, vote out existing members and shut down the entire testnet within relatively short period of time.

1 Like
#9

Awesome - I think that would be great actually.

Then in v2 we put in more safeguards. Perhaps we experiment with staking identities in an automatic manner - like Keybase, onename, etc… Or we decide that we do something like Dribbble with sponsoring… We still can keep it open and permissionless to onboard - but create anti-spamming safeguards. That’s the beauty of this tech - it is about controlling spam through a fine tuning of incentive structures.

Again, the type of failure that you bring up would force us to understand different failure modes better.

2 Likes
#10

Allow me to rephrase / reframe.

We are starting up a new restaurant and would like to figure out if our systems are going to work. Wouldn’t you want to see if the wait staff, kitchen, and all the other systems of your restaurant are working like you think they should?

But you only allow the public to come and sit in the restaurant - a smaller set of vet’ed folks (i.e. people that got through some restrictions) were allowed to actually order. Sure, these extra people sitting would allow you to see what the place looks like packed and understand the exact topology of the restaurant - but you don’t really know how the place will behave when it gets slammed. Or when someone gets too drunk and vomits everywhere…

Saying that anyone can be a bootnode is sort of like that. Sure, there’s a lot of people - but are they really pushing the systems to their limits? Is having perfect actors (participants) ideal? I don’t believe so.

1 Like
#11

Well, this attack can come at anytime and can block our ability to test existing functionality as well as projects that are in development. Spinning a new testnet is a very time consuming and expensive process. I would like to hear an opinion from POA foundation team if they are ready to be exposed to this attack.

1 Like
#12

Agreed.

I think that perhaps we might need to consider these costs. Another aspect of this tech is that one must create sustainable solutions. We might come to the conclusion that validators will need to price in support infrastructure. (Again - nothing is “free”)

1 Like
#13

We can create a clone POA network and test these ideas there while keeping Sokol very similar to core.

2 Likes
#14

Jeff, do you expect Core network to have those properties what you just described? Auto-inclusion, openness, having 100+ validators any time soon. Isn’t that what test network if for? To experiment and test the potential properties of the Core network, that is the sole purpose of the test network in my opinion…

As to the auto-inclusion, the essential part of the governance is voting. If you allow auto-inclusion you take that ability off of the governance to function at its fundamental level.

2 Likes
#15

We are talking about the testnet.

#16

Good idea! Jim and I are working on testing bridges and creating POA Clone network(s), we could later repeat the same process on different hardware where more people could have access and we could run several experiments in parallel if needed.

2 Likes
#17

I am talking about testnet too as a playground for the Core network. If you never expect those properties to transfer to the Core network then you are proposing a waste of resources. Clone networks is a great idea in this case.

2 Likes
#18

I still believe there needs to be a pre-Sokol step that exposes potential validators to any offchain aspects of the governance model we ultimately propose. It is becoming increasingly likely that offchain governance will eventually wind up being a significant component of the validator role. It can also get people involved during the notary application process.

I also understand Jeff’s concerns about vetting, but I believe those can be alleviated by sufficiently distributing the vetting process across multiple validators. For example, each candidate needs to be vetted by X different validators, maybe even selected at random.

2 Likes
#19

Rocco, I like your idea of having people involved with the community and other members on early stages and during the notary applications process. We have objective metrics on our forum that show a level of involvement. One of them is the Member badge. We can use this metric as one of the on-boarding steps. What do you guys think?

00%20AM
Screen Shot 2018-02-22 at 9.56.00 AM.png778×142 16.2 KB

1 Like
#20

I believe a list of requirements to become Sokol validator could be:

  • Actively participate in forum discussions
  • Run atleast one bootnode on either Sokol or Core networks for N period of time
  • Be accepted to Sokol via governance process (by Sokol validators)
  • Notary license. If pending license is allowed, I am not sure what would be the process to verify that. For new Notary candidates, it could be a certificate of completion 6 hours course, but most likely in some of the states 6 hours course is not required. Would like to hear your thoughts on how to verify pending notary license process
  • Not required, but desired: participate in Github activity. Actively test different scenarios for new projects, follow procedures and provide feedback, etc.
  • Submit introduction to POA forums

Any other items that could be added here? (Or removed)

Now regarding pre-acceptance process:

  1. Sokol validator candidate should meet all requirements
  2. One of active Sokol (and Core?) validators should propose adding this candidate as validator on Sokol network. This could be done by posting introduction to telegram Sokol Support channel (or new communication channel that we need to come up with).
  3. One of Sokol validators should create new ballot to add this candidate
  4. All active Sokol validators should cast a vote
  5. If majority voted yes, new candidate should be added to Sokol validators channel (whatever that would be)
  6. Deploy validator node
1 Like