Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

获取顶点 Shard信息接口报异常,提示”The length of array should be equal“ #1299

Closed
tmljob opened this issue Dec 16, 2020 · 5 comments
Labels
bug Something isn't working

Comments

@tmljob
Copy link
Contributor

tmljob commented Dec 16, 2020

Expected behavior 期望表现

通过” 获取顶点 Shard信息“restful接口获取分片信息,正常返回结果。

GET http://10.19.151.133:8980/graphs/daas/traversers/vertices/shards?split_size=67108864

Actual behavior 实际表现

接口报错,提示如下异常:

{
    "exception": "class java.lang.IllegalArgumentException",
    "message": "The length of array should be equal",
    "cause": ""
}

Steps to reproduce the problem 复现步骤

shard-api

Specifications of environment 环境信息

server0.11.2+hubble1.5.0+backend-hbase

@javeme javeme added the bug Something isn't working label Dec 17, 2020
@zhoney
Copy link
Contributor

zhoney commented Dec 17, 2020

@tmljob
使用hbase shell执行scan 'hbase:meta'获取下g_v表的regioninfo信息(如果有敏感信息可以脱敏一下)

@tmljob
Copy link
Contributor Author

tmljob commented Dec 17, 2020

@tmljob
使用hbase shell执行scan 'hbase:meta'获取下g_v表的regioninfo信息(如果有敏感信息可以脱敏一下)

scan_daas.zip

详见附件。

说明:这套hbase上,我们分别用0.10.4(graph-name:hugegrah)&0.11.2(graph-name:daas)建了两个图库,0.10.4版本的shard api可以正常请求,而0.11.2版本的shard api请求报如上错误。附件是我执行所述命令后,过滤出daas图库的相关信息。

@zhoney
Copy link
Contributor

zhoney commented Dec 21, 2020

@tmljob 在提供的信息中有如下一行

 daas:g_v,\x915000w\xE6\x9D\x8E\xE5\x9B\x9B1594131,1608016600046.bb50a240e1610d895ecc9880f1325061. column=info:regioninfo, timestamp=1608110184101, value={ENCODED => bb50a240e1610d895ecc9880f1325061, NAME => 'daas:g_v,\x915000w\xE6\x9D\x8E\xE5\x9B\x9B1594131,1608016600046.bb50a240e1610d895ecc9880f1325061.', STARTKEY => '\x915000w\xE6\x9D\x8E\xE5\x9B\x9B1594131', ENDKEY => '\x925000w\xE6\x9D\x8E\xE5\x9B\x9B11000763'}

我用hbase shell转为byte[]时,发现一个奇怪的现象:

hbase(main):015:0> Bytes.toHex(Bytes.toBytesBinary("\x915000w\xE6\x9D\x8E\xE5\x9B\x9B1594131"))
=> "fd35303030774edb31353934313331"
hbase(main):016:0> Bytes.toHex(Bytes.toBytesBinary("\x925000w\xE6\x9D\x8E\xE5\x9B\x9B11000763"))
=> "fd35303030774edb3131303030373633"
hbase(main):020:0> Bytes.compareTo(Bytes.toBytesBinary("\x915000w\xE6\x9D\x8E\xE5\x9B\x9B1594131"), Bytes.toBytesBinary("\x925000w\xE6\x9D\x8E\xE5\x9B\x9B11000763"))
=> 4

也就是startkey大于endkey!

这是HugeGraph在计算hbase的region分区时报错的原因

@tmljob
Copy link
Contributor Author

tmljob commented Dec 22, 2020

这个是怎么产生的呢?这个daas库里面的数据,是两种方式产生的:
a)初始化完成后,先通过hubble的gremlin界面,创建了部分测试数据;
b) 然后通过hugegraph-tool-1.5.0的图迁移migrate,把0.10.4的hughgraph图库的内容迁移到0.11.2的daas图库。

我又试了下边的shard api,可以查询,但结果有点奇怪,有的分片的起始&结束标识是空的。

@zhoney

1、split_size=67108864

GET http://10.19.151.133:8980/graphs/daas/traversers/edges/shards?split_size=67108864

结果如下:

{
    "shards": [
        {
            "start": "jzEwMHflvKDkuIk0NjAwNQ==",
            "end": "lz/1490MLyevz+0eMlFRVQ==",
            "length": 0
        },
        {
            "start": "lz/1490MLyevz+0eMlFRVQ==",
            "end": "n067l0Iyoa5651EILnJydQ==",
            "length": 0
        },
        {
            "start": "n067l0Iyoa5651EILnJydQ==",
            "end": "p12BSqdZFDVF/rTyKpOTlQ==",
            "length": 0
        },
        {
            "start": "p12BSqdZFDVF/rTyKpOTlQ==",
            "end": "r2xG/gx/hrwRFhjcJrS0tQ==",
            "length": 0
        },
        {
            "start": "r2xG/gx/hrwRFhjcJrS0tQ==",
            "end": "t3sMsXGl+ULcLXzGItXV1Q==",
            "length": 0
        },
        {
            "start": "t3sMsXGl+ULcLXzGItXV1Q==",
            "end": "v4nSZNbMa8mnROCwHvb29Q==",
            "length": 0
        },
        {
            "start": "v4nSZNbMa8mnROCwHvb29Q==",
            "end": "x5iYGDvy3lByXESaGxgYFQ==",
            "length": 0
        },
        {
            "start": "x5iYGDvy3lByXESaGxgYFQ==",
            "end": "z6ddy6EZUNc9c6iEFzk5NQ==",
            "length": 0
        },
        {
            "start": "z6ddy6EZUNc9c6iEFzk5NQ==",
            "end": "17YjfwY/w14IiwxuE1paVQ==",
            "length": 0
        },
        {
            "start": "17YjfwY/w14IiwxuE1paVQ==",
            "end": "38TpMmtmNeTTonBYD3t7dQ==",
            "length": 0
        },
        {
            "start": "38TpMmtmNeTTonBYD3t7dQ==",
            "end": "59Ou5dCMqGueudRCC5yclQ==",
            "length": 0
        },
        {
            "start": "59Ou5dCMqGueudRCC5yclQ==",
            "end": "7+J0mTWzGvJp0TgsB729tQ==",
            "length": 0
        },
        {
            "start": "7+J0mTWzGvJp0TgsB729tQ==",
            "end": "9/E6TJrZjXk06JwWA97e1Q==",
            "length": 0
        },
        {
            "start": "9/E6TJrZjXk06JwWA97e1Q==",
            "end": "////////////////////9Q==",
            "length": 0
        },
        {
            "start": "////////////////////9Q==",
            "end": "",
            "length": 0
        },
        {
            "start": "",
            "end": "R5iYGDvy3lByXESaGxgYGg==",
            "length": 0
        },
        {
            "start": "R5iYGDvy3lByXESaGxgYGg==",
            "end": "jzEwMHflvKDkuIk0NjAwNA==",
            "length": 0
        },
        {
            "start": "jzEwMHflvKDkuIk0NjAwNA==",
            "end": "jzEwMHflvKDkuIk0NjAwNQ==",
            "length": 0
        }
    ]
}

2、split_size=4294967295

GET http://10.19.151.133:8980/graphs/daas/traversers/edges/shards?split_size=4294967295

结果如下:

{
    "shards": [
        {
            "start": "jzEwMHflvKDkuIk0NjAwNQ==",
            "end": "",
            "length": 0
        },
        {
            "start": "",
            "end": "jzEwMHflvKDkuIk0NjAwNQ==",
            "length": 0
        }
    ]
}

@zhoney
Copy link
Contributor

zhoney commented Dec 23, 2020

@tmljob

我又试了下边的shard api,可以查询,但结果有点奇怪,有的分片的起始&结束标识是空的。

这个是预期的,当region的startkey或者endkey本身为空时,对应的shard的start或者end就会为空字符串

@zhoney zhoney closed this as completed in 0f15c46 Dec 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants