There is a lot of misunderstanding surrounding the concept of signers, scopes and witnesses in Neo N3. Some try to compare it with how other blockchains operate, some think it’s not very useful, some just don’t know what it provides to them. This article tries to explain how exactly this part of Neo N3 operates. It’s mostly intended for dApp developers, but curious users can also get a better understanding of why Neo N3 is good for them.
Neo N3 transaction
First, let’s take a look at what exactly is a Neo N3 transaction.
We won’t dig into every field of it, they all of course have some purpose and meaning behind them, instead let’s concentrate on things that matter for our discussion:
- script is a NeoVM bytecode, also known as ”entry script”, this is what gets executed when transaction is processed by nodes
- signers is an array of accounts (Uint160 script hashes) with scopes
- witnesses is an array (with the number of elements exactly matching the number of signers) of script pairs containing verification script (that hashes to signer script hash) and invocation script (that usually contains signatures checked by verification script)
So transaction can be seen as a byte code container signed by a number of signers with each of them having some signature scope. This structure is quite different from many other blockchains, it doesn’t have a single sender or receiver (which also is the reason for ”from” parameter in NEP-17 ”transfer” method and other witness-requiring methods like ”vote” in NEO token) and every transaction actually bears some Turingcomplete bytecode with it. Combination of these factors makes Neo N3 transactions very powerful.
Whereas in other blockchains almost any non-trivial action might require deploying a contract, Neo allows to have any kind of logic in transaction itself (like calling multiple contracts or the same contract multiple times in a single transaction). And having multiple signers simplifies multi-party interactions a lot, again a single transaction with several signers can substitute a set of transactions with additional on-chain contracts used by other chains.
There is another degree of flexibility here in that signers are identified by script hashes and this gives a full power of NeoVM to every signer verification script. While this is not a unique feature of Neo, many newer blockchains tend to simplify this scheme down to trivial single-key signature checks justifying it by ”optimization”. That’s a bit different subject, but just keep in mind that NeoVM is there for you even in this case.
If we’re to look at transaction as some kind of task for blockchain to process, the difference between ”from-to-data” and ”script-signers” schemes can be summarized as the difference between ”tell the receiver that there is some data for him from this key” and ”execute some code signed by these guys with these scopes”. While the former makes for a nice messaging system (that for some strange reason does code executions as a side-effect for some messages), the latter is what can drive much more complex scenarios.
And still, developers working with smart contracts on Neo N3 platform know that if they’re to retrieve transaction from the Ledger contract or from the script container, they can clearly see a ”Sender” field with no scope data. So this of course needs an additional explanation.
Sender
Unlike most of Legacy transactions Neo N3 transactions are always paid for, there are no free ones and in a system where we legitimately can have a number of signers an obvious question is — who pays the bills? And this ”sender” notion is the answer to that. Structurally it’s just the first signer in a list of signers. It’s his GAS balance that the fees will be deducted from when block with transaction is to be processed.
Does the sender sign a transaction? Of course he does, there is no way around it, his valid signature is 100% there the same way any other signer’s signature. Does it mean that the sender witnesses and allows any action to be performed in any invoked contract in this transaction? No, and that’s the key.
Being a sender means paying the bills and nothing more; this implicit role can’t be compared to sender’s powers of other blockchains. In fact, Neo N3 allows one to be a sender and give no permissions at all, paying the fee but not allowing any actions requiring an account witness.
The other common misconception (probably based on experience with messagebased blockchains) is that ”Sender” is the same as ”Invoker” which is what happens when contracts call each other by passing messages. This is not the case with Neo, it has proper invocation stack in NeoVM and at any allowed depth level the sender you get for transaction is the same (as transaction is the same); retrieving the script hash of the calling contract is possible, but there is a specific interface for that (”GetCallingScriptHash”).
These are the reasons proper Neo N3 contracts shouldn’t be relying on sender account they can get for any kind of security purposes. Practically if contract looks at this field at all it’s very likely to be making some kind of mistake, because witnesses are to be checked with ”CheckWitness” call and sender hash doesn’t have a lot of use to it.
CheckWitness
So what exactly ”CheckWitness” does and what’s so special about it? Way back when in Neo Legacy it allowed to check a hash against a list of script hashes verifying this transaction. These hashes were taken from transaction inputs, attributes and other places depending on transaction type, so ”CheckWitness” basically unified all of these sources.
Neo N3 has much simpler transaction model, some of the old ”CheckWitness” logic is just not relevant, but in fact it has gotten even more powerful. The basic meaning of this function is the same, in general it answers the question of whether the account (script hash) approves (witnesses/verifies) modifying some state controlled by it. Usually it’s about asset movements, like NEP-17 transfer requires a witness from asset owner to perform transfer to another account. But different contracts can use it in different ways, for example NEO contract’s ”vote” method also requires an approval from voter even though it doesn’t change the balance of voter’s account. As accounts are identified by script hashes that’s what ”CheckWitness” accepts.
The first thing it does while performing the actual check is comparing this script hash to the script hash of the contract that has called the contract that is being currently executed. The presumption here is that when some contract calls another one it gives an automatic witness for this call because if it doesn’t want for the call to succeed it might as well not do it at all. This simple ”CheckWitness” logic allows contracts to do anything possible with regular accounts, this is what makes NeoBurger voting possible, for example.
Then ”CheckWitness” goes through signers list trying to find the requested hash. If it’s not there, there is no witness provided and the check fails. If the hash is present in the list then it compares witness scope for this signer against current execution context, if everything goes well it returns ”true”, but if the scope doesn’t fit the check will fail. Notice that it uses a whole list of signers with each of them having a scope, this logic can’t be replicated by a smart contract in any way, so correctly using ”CheckWitness” for approval is an important aspect of Neo N3 contract development.
Signature scopes
Here is a list of scopes that can be used by signers:
- None
- CalledByEntry
- CustomContracts
- CustomGroups
- Rules (since 3.1.0)
- Global
They can be combined (except for ”None” and ”Global”), and there are reasons and use cases for all of them. Let’s walk through the list, we’ll use single signer in our examples to simplify things, but having a number of them doesn’t change anything scope-wise.
Global
Let’s start exploring scopes with the simplest one — ”Global”. Some might argue that ”None” is even simpler, but that’s not exactly the case. We can say that ”Global” predates it because that’s the way Legacy chain witnesses worked, authorizing any action related to account in question in any contract invoked from a transaction. Sounds perfect, the transaction is signed, we know the signature is correct, and numerous other blockchains use similar model where this signature approves anything done to the account. And yet Neo N3 introduced a number of additional scopes, why is that?
The main reason is security and safety of transaction processing, especially in presence of malicious and/or broken contracts. If we’re to suppose that all contracts are perfect and the user only calls these perfect contracts from the entry script, then using ”Global” wouldn’t be a problem. But reality differs a bit from this ideal world.
There’ve been cases on Legacy network as well as on non-Neo chains where this interpretation of witnesses has led to loss of assets. For example, someone created a fake token with the same symbol and name as some legitimate one. This token technically is a contract which has a different hash from the original one, but then the creator of this token can airdrop arbitrary amounts of it to different addresses. Then users receiving this token can either confuse it with the original one or try to interact with it just out of curiosity. And that’s where the danger comes from — they’re invoking some seemingly benign method of this malicious token (like ”transfer”), but instead of transferring its own tokens the contract invokes NEO/GAS/any other token contract and transfers these tokens to itself or some other predefined address controlled by the malicious actor.
In this case with ”Global” signature scope (or complete lack of scoping) good contracts can’t differentiate legitimate calls from bad ones, it’s all ”transfer” calls in a transaction signed by the account the transfer is requested from. Whether it’s a direct call from the entry script or indirect one from the malicious contract doesn’t matter, there are cases where this behaviour (contract requesting transfer from user) can be valid and there are ones where it can’t. This scenario can be further extended if we’re to talk about multiple parties signing a single transaction, having approval for any of them makes such attacks even worse than they’re for a single account.
This original problem was one of the main triggers for signature scoping mechanism that Neo N3 has implemented. While ”Global” is still available for you to use, it’s not recommended, it’s hard to imagine any case that couldn’t be solved with more strict and safe scopes now. Let’s delve into these new possibilities.
None
A complete opposite of ”Global” is ”None”, it means that this signer forbids using his signature in any contract. Sounds counterproductive — why would he sign the transaction then? But in fact it’s very helpful when used for signers in sender role, with this scope they only pay the fees, but not do anything else.
It may seem an obscure case, but if you’ve used Oracle subsystem of Neo N3, you have also used ”None” witness scope. The oracle response transaction containing an answer to oracle request has two signers: the first one is the oracle native contract which holds GAS it accepted with an oracle request and the second one is oracle nodes multisignature contract (and also ”None” scope!). So we have oracle contract paying for the response transaction, but not allowing any other actions for this account and if you think about it for a while you may notice that this scheme wouldn’t be possible at all with ”Global” scope.
An oracle response transaction calls oracle contract first and then this contract invokes the callback specified when request was created. This callback in general can be anything, if there was no signature scoping someone could easily invoke ”transfer” method of GAS contract to get tokens belonging to oracle contract account and GAS couldn’t resist giving them to the caller, because oracle contract is one of the signers. But with ”None” scope ”CheckWitness” call in GAS contract fails, so it’s not possible (there are other oracle-specific reasons that make it impossible too, but that’s a bit different topic). So ”None” scope allows for safe transaction sponsoring in general, contract- or account-based, the sponsor can be sure that his assets are safe, and the only thing he does is paying fees with his GAS.
However, oracle response also contains another signer, also with ”None” scope. It may seem even more strange to have it, because it technically does nothing; ”CheckWitness” never works for it and it’s not involved in fee payments, and yet it’s an important part of the whole oracle scheme. That’s because an oracle response must be signed by oracle nodes, a specific set of keys stored in the ”RoleManagement” native contract. It’s a multisignature account requiring three out of four signatures for current public networks, even though it doesn’t normally have any assets an oracle response transaction shouldn’t allow messing with it. That’s also ensured by ”None” scope, the signature is there, it can be checked, but ”CheckWitness” will never allow doing anything with this account in such a transaction.
Simple as it is, ”None” is a very effective scope for such special cases.
CalledByEntry
This scope is a safe default covering most of the cases, that’s what you’ll be using most of the time when dealing with Neo N3. The permission is given to the entry script and ones it directly calls. Most of invocations do not require going deeper than that and even if some additional contracts are being called they might not require witnessing any actions (like calling ”StdLib” functions). An entry script is created by a user (OK, fancy UI controlled by a user), so user should trust it and it directly contains code that calls some other contract or contracts, so these calls are well known, user is likely to want them to happen and be authorized.
Notice how this simple restriction prevents the bad case described for ”Global” scope. If someone invokes bad contract with ”CalledByEntry” scope the permission would only be given to this contract itself. If it tries to steal user’s GAS/NEO/any other token that’d be at least one additional call down the invocation stack and ”CheckWitness” doesn’t work there for user’s account with ”CalledByEntry” scope.
In vast majority of the cases this scope is all you need. But there are more complex scenarios, ones where you have a set of interacting contracts requiring witnesses and that’s where other scopes might be handy.
CustomContracts
”CustomContracts” allows one to specify exactly the contract or set of contracts the witness is valid for. These allowed contracts are specified as script hashes and if ”CheckWitness” is called from one of them, it succeeds; if some other contract is to call it, it’ll fail.
It’s a nice addition to ”CalledByEntry” for cases with additional internal invocations from the contract called by the entry script, but they also need a witness. The user (frontend code) is supposed to know which contract is going to be called and add it to the list of witnessed contracts. If something goes wrong and an unexpected contract is called instead of the appropriate one the user is protected because it won’t have his witness.
CustomGroups
Very similar to ”CustomContracts” this scope allows to add witness for a group (or a number of groups) of contracts. Groups are a NEP-15 concept, they’re specified in contract’s manifest and, in general, a contract can belong to a number of groups. Each group is identified by a public key, and this association is confirmed by a signature verifiable using this public key (so Joe Random’s contract can’t be a part of the group unless he can forge a signature for it).
Grouping itself is not used very often, but there are cases where it’s very handy — a complex dApp can be split into a set of interacting contracts and then this set can be grouped using a single key to sign manifests. Transactions interacting with this dApp can then give witness to a whole group, and irrespective of how contracts call each other inside of the group everything will work smoothly.
Rules
It may seem that the set of scopes described above covers everything and in fact that was the set of scopes available at the time of the official Neo 3.0.0 release. But version 3.1.0 (and compatible neo-go 0.98.0) has introduced a new one: ”Rules”, and it has an interesting story behind it.
Notice that ”CustomContracts” and ”CustomGroups” scopes limit witness validity to some set of contracts or groups, but these contracts or groups may appear anywhere in the invocation stack. This leaves some potential for reentrance attack, especially given the NEP-11/NEP-17 ”onNEPXXPayment” functionality. Trusted (and witnessed) contract can call another contract which in turn can call trusted contract again, and it will get a valid witness even though it was never intended to.
There were various proposals to fix this, but the resulting mechanism we have now is ”Rules” scope, which technically can replace any of the above (but they of course are still available both for compatibility and simplicity reasons). ”Rules” scope is exactly that — a set of rules with predicates matching some condition and action performed if the match is successful. It can be seen as a set of firewall rules that are being examined one by one until some of them matches and returns a result.
The result, unlike with previous scopes, can be poth positive and negative, it’s also known as ”action”: ”allow” or ”deny”. If rule isn’t matched, it’s action is ignored, but to perform a match one writes a boolean expression using the following available conditions:
- Boolean: true or false, mostly useful for testing or emulating Global/None scopes
- Not: inverting nested condition
- And: matching a whole set of nested conditions (up to 16 of them)
- Or: matching one of conditions from nested set (also up to 16)
- ScriptHash: contains a hash to compare with script that is being currently executed (similar to CustomContracts)
- Group: contains a key identifying a group to compare with groups of the currently executing script (similar to CustomGroups)
- CalledByEntry: evaluates to true if the script is an entry script or one called directly by it (like CalledByEntry scope)
- CalledByContract: contains a hash to compare with the caller script hash
- CalledByGroup: contains a key identifying a group to compare with the groups of the caller script
This is not as simple as previous scopes, and yet it gives a lot of expression power to users and developers creating transactions. The case with reentrancy can now be easily solved by combining (And) checks for current group (Group) and calling group (CalledByGroup) or hash (CalledByContract) in a single rule. Notice that rules can be nested in some cases, but the depth of this nesting is limited to just two levels at the moment, which practically is enough given that you can add more rules (up to 16).
Conclusion
Scoped witnesses is one of the distinct core features of Neo N3. Deployed contracts can contain any code and invoke other contracts as they want, but with scoped signatures users now have control over what they approve to be done in their transactions and what not. Remember that the power of scopes is there, check witnesses correctly contractside, control what witnesses you give to whom on the user side — and the system will make your interactions much safer.
References
- Transaction details: https://docs.neo.org/docs/en-us/basic/concept/ transaction.html.
- Syscalls: https://docs.neo.org/docs/en-us/reference/scapi/interop.html.
- GH issue on voting from contract: https://github.com/neo-project/neo/issues/1924.
- CheckWitness documentation: https://docs.neo.org/docs/en-us/develop/write/basics.html#checkwitness.
- GH issue on scoped witnesses: https://github.com/neo-project/neo/issues/544.
- NEP-15: https://github.com/neo-project/proposals/blob/master/ nep-15.mediawiki.
- NEP-17: https://github.com/neo-project/proposals/blob/master/nep-17.mediawiki
- GH issue on Custom* scopes: https://github.com/neo-project/neo/issues/2583.
© NeoSPCC, CC BY-SA 4.0
