Cannot replicate LCS Serialization for AccountAddress in C# (Net Core)

dbascans · September 25, 2019, 12:47am

Hello, I’m trying to understand the new LCS Serialization, reading the [Libra Canonical Serialization (LCS)] (https://github.com/libra/libra/blob/d54980c7fd8d40c9c3d215399b23bd55c263a188/common/canonical_serialization/README.md)
(Thanks @Rio900 for that)

But I can’t even replicate the most basically example.

How I can get the size of the address, that there says is 32 but I think is 64, if I count the address string.

Adding a code snippet to helps to understand what I meant. Trying to replicate the same as the image,

 string Address = "ca820bf9305eb97d0d784f71b3955457fbf6911f5300ceaa5d7e8621529eae19";

        string LCSSerializarion = "[" +
            Address.Length +  // HERE I DONT KNOW WHAT TO DO
            Address.ToUpper() + "]";

I know that is a dummy question but I’m blocked here, any help is appreciated!

Rio900 · September 25, 2019, 5:43am

Hello @dbascans , I quickly created an example, it will be useful for you. I would really appreciate If you finish Deserialization, it would be great for all of us

About your question: BitConverter.GetBytes(Address.Length).Concat(Address.HexStringToByteArray());

(region LCS test)

github.com

Rio-dapp/libra-csharp-sdk/blob/master/Examples.CLI/Program.cs

using LibraAdmissionControlClient;
using System.Linq;
using System;
using Types;
using Grpc.Core;
using static AdmissionControl.AdmissionControl;
using LibraAdmissionControlClient.Dtos;
using LibraAdmissionControlClient.LCS.LCSTypes;
using LibraAdmissionControlClient.LCS;
using NSec.Cryptography;
using System.Text;

namespace Examples.CLI
{
    class Program
    {
        static void Main(string[] args)
        {
            LibraAdmissionControl service = new LibraAdmissionControl();

This file has been truncated. show original

dbascans · September 25, 2019, 12:14pm

Nice! Thanks for the example!
It’s my idea to end all the serialization and deserialization.
But yesterday I was comparing your serialization with the documentation example, and there is some differences in the names and symbols (I could share my code if you want it, maybe Im doing everything wrong).

Yesterday I was working to replicate the models that are showed in the documentation, is not very clear to me yet (I will be still working on that today).

For example, how we can calculate the length in the response to deserialize the LCS, I assume that it will come a response same as we send the request (Length + Data).

Other question that is not in the documentation is how now we will sign our transactions?, I could find that in the documentation, but maybe is other documentation somewhere.

Thanks for your awnsers, you have been very helpful !

autsada · September 25, 2019, 1:05pm

@Rio900, I posted here if you could help. I created serialize function in javascript and it gets LCS bytes correctly (checking the result from my function against the example in the README document ). But when created the signed transaction that used to submit to the testnet, I got “submit txn rejected”. I couldn’t find out what wrong in my code.

Given below data to create peer to peer transaction.

const rawTxn = {
code: peerToPeerCode.code,
args: [
  { amount: 8000000 },
  {
    address:
      '5d43fff83cfde40ac05c91575185bef36a714a93f86a2e799d4ee3c40185443c'
  }
]
  }

const senderAddress = 'f5cddc6cf9f3a9f49d01dd1e976ac5947d4cfe030403f1b07402076d94065266'
const senderPublicKey = 'ba6cfe1913b3b75a31b63e16794677fef5a1d21607c7302d6046e2bea6a9df4c'
const senderPrivateKey = '9ddcfece360ddfc5de6ca822b0b1a484b9fe48cc8a8933761af5b89f0629b4971125c40c8e8044aef8dc32e5b9134852c9ed758bf677be2752ed9bd35b4e007f'

const MaxGasAmount = 140000
const GasUnitPrice = 0
const SequenceNumber = 0

How should LCS representation of the signed transaction of the above raw data looks like?

Rio900 · September 25, 2019, 6:23pm

@dbascans, I’d check out your code, if you share it with me.
You’re right, usually, Libra uses Length+Data, but not for all types of objects (every object should be reviewed as a separate case). When I implemeted LCS in SDK, I used only Libra ReadMe as a documentation.

Rio900 · September 25, 2019, 6:27pm

@autsada , could you please copy your raw transaction bytecode that you sign? (I’ll try to deserialize it and check)

dbascans · September 26, 2019, 12:43am

Before I start, I’m just wondering, if the names are alright talking about Serialization a Deserialization, because I think that now I’m trying to Serialize not to Deserialize, but that are just names, if you can explain that is better

Well I started from the basics ones. Following the examples in the Readme.md and trying to get the same result.

I just could deserialize two of all of them. I don’t know why, the main problem that I’m facing is the prefix in all of the transactionArguments.

I started by the Address, then the U64 Transaction Argument, and now I’m trying to do the same with the String transactionArgument.

github.com

1950Labs/1950Libra_client/blob/master/LibraClient/LibraClient/Program.cs

using AdmissionControl;
using Google.Protobuf;
using Grpc.Core;
using LibraClient.Enums;
using LibraClient.LCSTypes;
using NBitcoin.DataEncoders;
using NSec.Cryptography;
using SHA3.Net;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
using Types;

namespace LibraClient
{
    class Program
    {

This file has been truncated. show original

As you can see, I adapt your code to a simply console application.

When I’m trying to Serialize (Or deserialize) the string Im getting.

0d00000048656c6c6f2c20576f726c6421

while the Documentation gets.

020000000D00000048656C6C6F2C20576F726C6421

As you can see the prefix is the wrong part here since the last part of the string is fine (I think)

48656c6c6f2c20576f726c6421

Tell me if I’m doing this stuff OK, or I should that another approach!

Thanks for all the help!

autsada · September 26, 2019, 1:01am

@Rio900 Thank you very much, here is my raw txn bytes.

[20000000f5cddc6cf9f3a9f49d01dd1e976ac5947d4cfe030403f1b07402076d94065266000000000000000000000000b80000004c49425241564d0a010007014a00000004000000034e000000060000000d54000000060000000e5a0000000600000005600000002900000004890000002000000008a90000000f00000000000001000200010300020002040200030204020300063c53454c463e0c4c696272614163636f756e74046d61696e0f7061795f66726f6d5f73656e6465720000000000000000000000000000000000000000000000000000000000000000000100020004000c000c0113010102020000000000000000127a000000000001000000200000005d43fff83cfde40ac05c91575185bef36a714a93f86a2e799d4ee3c40185443c00000000e0220200000000000000000000000000e00a8c5d00000000]

Rio900 · September 26, 2019, 8:02pm

@autsada, i analyse your bytecode, it should be Script not the Program.
In testnet all peer to peer transaction are as a Script

Rio900 · September 26, 2019, 8:24pm

@dbascans, I gave names Serialization and Deserialization because Libra Canonical Serialization is (LCS), and i decided that it will be understandably, but as you had this question clearly I was wrong, I will rename them to Serialize and Deserialize

02000000 - this is arg type, if you need String it is 3
0D000000 - len

autsada · September 27, 2019, 2:54am

@Rio900 Thank you very much, I changed from program to script which I understand its type should be 2, but it still doesn’t work. This is the new raw byte code.

[20000000f5cddc6cf9f3a9f49d01dd1e976ac5947d4cfe030403f1b07402076d94065266000000000000000002000000b80000004c49425241564d0a010007014a00000004000000034e000000060000000d54000000060000000e5a0000000600000005600000002900000004890000002000000008a90000000f00000000000001000200010300020002040200030204020300063c53454c463e0c4c696272614163636f756e74046d61696e0f7061795f66726f6d5f73656e6465720000000000000000000000000000000000000000000000000000000000000000000100020004000c000c0113010102020000000000000000127a000000000001000000200000005d43fff83cfde40ac05c91575185bef36a714a93f86a2e799d4ee3c40185443c00000000e022020000000000000000000000000099778d5d00000000]

Still need to figure out more.

drussi · September 27, 2019, 1:30pm

just a quick reply in case it helps and this conversation is moving forward nicely I think.
An address gets serialized as a bytearray so the serialization is the size in LittleEndian and the bytes. For size we allow up to max_u32 so 4 bytes for the size. Given the Address is a 32byte “entity” it’ll be 20 00 00 00 32byte_address.
Strings are currently not well supported in Move so passing a String as an arg it is not going to do a lot of good!
Changing Program to Script is very good as Program is deprecated and about to be removed but it should not affect the result as of right now.
Hope this helps

Rio900 · September 27, 2019, 2:49pm

How you send Signature and SenderPublicKey through proto?
I see only SignedTxn property…

dbascans · September 28, 2019, 12:28am

Wow that helped me a lot.
FYI… I think that the TransactionArgumentEnum is not in order, because the serialization of the number 3 is 03000000 and for the 2 is 02000000, but I don’t know which one is the good one.

Well, I advanced until the Access Path, not really sure if im in the right path now, also with the ByteArray TransactionArgument

This is an example of the ByteArray Serialization.

TransactionArgumentLCS ByteArrayArgument = new TransactionArgumentLCS()
        {
            // TESTING BYTE ARRAY
            ByteArray = Encoding.UTF8.GetBytes("01217da6c6b3e19f18"),
            ArgType = (uint)TransactionArgumentLCSEnum.ByteArray
        };

        byte[] ByteArrayArgumentByteLcs = LCSCore.LCSSerialization(ByteArrayArgument);

        Console.WriteLine("\nBYTE ARRAY ARGUMENT SERIALIZED - " + ByteArrayArgumentByteLcs.ByteArryToString());
        Console.WriteLine("BYTE ARRAY ARGUMENT DESERIALIZED - " + LCSCore.LCSDeserialization<TransactionArgumentLCS>(ByteArrayArgumentByteLcs));

I have no clue of how to use the one of the documentation. Even if we couldn’t is fine to do it like this ByteArray = Encoding.UTF8.GetBytes(“01217da6c6b3e19f18”),

Don’t know how to convert the 0xb"cafed00d" to byteArray

Or how to convert this one to byteArray (01217da6c6b3e19f1825cfb2676daecce3bf3de03cf26647c78df00b371b25cc97)
In order to Serialize it using the LCSSerializer, I had to use some custom byteArray.

I updated my code if anybody wanna check it.

github.com

1950Labs/1950Libra_client/blob/master/LibraClient/LibraClient/Program.cs

using AdmissionControl;
using Google.Protobuf;
using Grpc.Core;
using LibraClient.Enums;
using LibraClient.LCSTypes;
using NBitcoin.DataEncoders;
using NSec.Cryptography;
using SHA3.Net;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
using Types;

namespace LibraClient
{
    class Program
    {

This file has been truncated. show original

autsada · September 28, 2019, 1:31am

The signed transaction that I used to construct submit transaction request is created under this structure:[rawTxnBytes publicKeyLen publicKeyBytes signatureLen signatureBytes]

Here is an example of the signed transaction bytes.

[200000005d43fff83cfde40ac05c91575185bef36a714a93f86a2e799d4ee3c40185443c000000000000000002000000b80000004c49425241564d0a010007014a00000004000000034e000000060000000d54000000060000000e5a0000000600000005600000002900000004890000002000000008a90000000f00000000000001000200010300020002040200030204020300063c53454c463e0c4c696272614163636f756e74046d61696e0f7061795f66726f6d5f73656e6465720000000000000000000000000000000000000000000000000000000000000000000100020004000c000c0113010102020000000000000000127a00000000000100000020000000c618ddea3030b4f94152a20b5c31df2f6e0ac42ef2ecf0ecb6c091fc08f1ffce00000000e022020000000000000000000000000031118e5d0000000020000000c73ea27b62fa352dbc6323b885889d4236499d38064856d7a7b07db43f6c865240000000a6e1d286f99dff079630f6a491e8c512c74d367f55946675e6993090400ba4f1fa50b0113e74b5b6b2abcbd375d22c098fa71408cc5442617d1bc1df9aba930b]

Where the bold one is the public key and the italic is the signature.

Rio900 · September 28, 2019, 9:16am

Payload Script has another structure than Program, you need remove Modules

Script struct:
byte[] Code
TransactionArgumentLCS[] TransactionArguments

Rio900 · September 28, 2019, 9:29am

I done Deserialize and push on master, you are right about args type order , I fixed it. And had renamed methods to LCDeserialize and LCSerialize

SDK has method HexStringToByteArray() it return byte array for strings as the "cafed00d"

When i send sign transaction I have result { "vmStatus": { "majorStatus": "2" } }, I’ll fix it later

autsada · September 28, 2019, 12:29pm

Thank you, I have removed Modules. I think my issue may occur in the setting SubmitTransactionRequest step, not the LCS of the signed txn. I am checking it. Thanks a lot for your help.

autsada · September 28, 2019, 2:46pm

Just before I was giving up, finally I was able to fix this issue. It turned out that I didn’t update the proto files. So after using the latest proto files. The submission succeeded. For LCS of peer to peer transaction arguments, it is important that the address arg must come before u64 arg.

dbascans · September 29, 2019, 12:36am

After trying a lot I finally made a transaction, and it seems that is working properly! (THANKS A LOT FOR ALL YOUR HELP)

I have updated my code, if you want to check my example, maybe you can notice what is the problem in your code

But (there is always a but), now I cannot deserialize the response, probably I’m not deserializing the the right object.

Actually Im trying to deserialize the SignedTransactionWithProof and the property of that object is SignedTransaction (SignedTrx) but I don’t know if that is the object that I have to deserialize or it’s other.

Also I created a new object SignedTransactionLCS in order to try to deserialize it, but when I tried to get the first array, the cursor of the deserializer returns me a pretty big number.

This is the code.

This is the inside of the response.

Thanks again for all your help! You have been pretty helpful!