Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] binlog load 方式同步数据,报 errCode = 2, detailMessage = transaction not found异常 #8171

Closed
3 tasks done
junjie4585 opened this issue Feb 21, 2022 · 3 comments

Comments

@junjie4585
Copy link

junjie4585 commented Feb 21, 2022

Search before asking

  • I had searched in the issues and found no similar issues.

Version

0.15

What's Wrong?

fe.conf
sys_log_level = WARN
http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010
mysql_service_nio_enabled = true

priority_networks = 172.16.10.0/24

enable_create_sync_job=true
enable_batch_delete_by_default=true
enable_http_server_v2=true
lower_case_table_names=1

be.conf
sys_log_level = WARNING

be_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060

priority_networks = 172.16.10.0/24

table
CREATE TABLE bm_grid_dor (
grid_id bigint(20) NOT NULL ,
grid_code varchar(50) ,
gb_code varchar(50) DEFAULT NULL ,
parent_id bigint(20) DEFAULT NULL ,
parent_code varchar(50) DEFAULT NULL ,
grid_classification varchar(10) NOT NULL ,
grid_level varchar(10) NOT NULL ,
grid_property varchar(10) DEFAULT NULL ',
is_end char(1) NOT NULL ,
grid_address varchar(200) DEFAULT NULL ,
grid_introduce varchar(1024) DEFAULT NULL ,

) ENGINE=OLAP
UNIQUE KEY(grid_id)
COMMENT "OLAP"
DISTRIBUTED BY HASH(grid_id) BUCKETS 3
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"in_memory" = "false",
"storage_format" = "V2"
);
sync jobs
CREATE SYNC social_system.sync_bm_grid
(
FROM social_sys.bm_grid INTO bm_grid_dor
)
FROM BINLOG
(
"type" = "canal",
"canal.server.ip" = "172.16.10.36",
"canal.server.port" = "11111",
"canal.destination" = "example",
"canal.username" = "canal",
"canal.password" = "canal"
);
canal instance.properties
canal.instance.mysql.slaveId=12341
canal.instance.gtidon=false

canal.instance.master.address=172.16.10.106:31016
canal.instance.master.journal.name=
canal.instance.master.position=
canal.instance.master.timestamp=
canal.instance.master.gtid=

canal.instance.rds.accesskey=
canal.instance.rds.secretkey=
canal.instance.rds.instanceId=

canal.instance.tsdb.enable=true
canal.instance.dbUsername=system
canal.instance.dbPassword=CCYdPAWF%Na8^roa
canal.instance.connectionCharset = UTF-8
canal.instance.defaultDatabaseName=social_sys
canal.instance.enableDruid=false

canal.instance.filter.regex=.\..
canal.instance.filter.black.regex=mysql\.slave_.*

canal.mq.topic=bm_mark_topic
canal.mq.partition=0
#################################################

What You Expected?

实现使用binlog load方式同步数据

How to Reproduce?

Java8编译,
系统版本:Linux localhost.localdomain 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
canal:0.

Anything Else?

主库手动增加3条数据,fe 错误日志:

2022-02-21 17:05:21,160 WARN (thrift-server-pool-12|237) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 3: errCode = 2, detailMessage = transaction not found
2022-02-21 17:06:03,062 WARN (thrift-server-pool-13|241) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 4: errCode = 2, detailMessage = transaction not found
2022-02-21 17:06:13,548 WARN (thrift-server-pool-11|193) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 5: errCode = 2, detailMessage = transaction not found

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@junjie4585
Copy link
Author

列映射问题,显示指定列映射才能正常导入

@yym12
Copy link

yym12 commented Mar 21, 2022

显示指定列也会报错
image

@junjie4585
Copy link
Author

需要把源表的所有字段都列到后面,即使目标表中不需要的字段也要在列映射上指定。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants