We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
0.15
fe.conf sys_log_level = WARN http_port = 8030 rpc_port = 9020 query_port = 9030 edit_log_port = 9010 mysql_service_nio_enabled = true
priority_networks = 172.16.10.0/24
enable_create_sync_job=true enable_batch_delete_by_default=true enable_http_server_v2=true lower_case_table_names=1
be.conf sys_log_level = WARNING
be_port = 9060 webserver_port = 8040 heartbeat_service_port = 9050 brpc_port = 8060
table CREATE TABLE bm_grid_dor ( grid_id bigint(20) NOT NULL , grid_code varchar(50) , gb_code varchar(50) DEFAULT NULL , parent_id bigint(20) DEFAULT NULL , parent_code varchar(50) DEFAULT NULL , grid_classification varchar(10) NOT NULL , grid_level varchar(10) NOT NULL , grid_property varchar(10) DEFAULT NULL ', is_end char(1) NOT NULL , grid_address varchar(200) DEFAULT NULL , grid_introduce varchar(1024) DEFAULT NULL ,
bm_grid_dor
grid_id
grid_code
gb_code
parent_id
parent_code
grid_classification
grid_level
grid_property
is_end
grid_address
grid_introduce
) ENGINE=OLAP UNIQUE KEY(grid_id) COMMENT "OLAP" DISTRIBUTED BY HASH(grid_id) BUCKETS 3 PROPERTIES ( "replication_allocation" = "tag.location.default: 3", "in_memory" = "false", "storage_format" = "V2" ); sync jobs CREATE SYNC social_system.sync_bm_grid ( FROM social_sys.bm_grid INTO bm_grid_dor ) FROM BINLOG ( "type" = "canal", "canal.server.ip" = "172.16.10.36", "canal.server.port" = "11111", "canal.destination" = "example", "canal.username" = "canal", "canal.password" = "canal" ); canal instance.properties canal.instance.mysql.slaveId=12341 canal.instance.gtidon=false
canal.instance.master.address=172.16.10.106:31016 canal.instance.master.journal.name= canal.instance.master.position= canal.instance.master.timestamp= canal.instance.master.gtid=
canal.instance.rds.accesskey= canal.instance.rds.secretkey= canal.instance.rds.instanceId=
canal.instance.tsdb.enable=true canal.instance.dbUsername=system canal.instance.dbPassword=CCYdPAWF%Na8^roa canal.instance.connectionCharset = UTF-8 canal.instance.defaultDatabaseName=social_sys canal.instance.enableDruid=false
canal.instance.filter.regex=.\.. canal.instance.filter.black.regex=mysql\.slave_.*
canal.mq.topic=bm_mark_topic canal.mq.partition=0 #################################################
实现使用binlog load方式同步数据
Java8编译, 系统版本:Linux localhost.localdomain 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux canal:0.
主库手动增加3条数据,fe 错误日志:
2022-02-21 17:05:21,160 WARN (thrift-server-pool-12|237) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 3: errCode = 2, detailMessage = transaction not found 2022-02-21 17:06:03,062 WARN (thrift-server-pool-13|241) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 4: errCode = 2, detailMessage = transaction not found 2022-02-21 17:06:13,548 WARN (thrift-server-pool-11|193) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 5: errCode = 2, detailMessage = transaction not found
The text was updated successfully, but these errors were encountered:
列映射问题,显示指定列映射才能正常导入
Sorry, something went wrong.
显示指定列也会报错
需要把源表的所有字段都列到后面,即使目标表中不需要的字段也要在列映射上指定。
No branches or pull requests
Search before asking
Version
0.15
What's Wrong?
fe.conf
sys_log_level = WARN
http_port = 8030
rpc_port = 9020
query_port = 9030
edit_log_port = 9010
mysql_service_nio_enabled = true
priority_networks = 172.16.10.0/24
enable_create_sync_job=true
enable_batch_delete_by_default=true
enable_http_server_v2=true
lower_case_table_names=1
be.conf
sys_log_level = WARNING
be_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060
priority_networks = 172.16.10.0/24
table
CREATE TABLE
bm_grid_dor
(grid_id
bigint(20) NOT NULL ,grid_code
varchar(50) ,gb_code
varchar(50) DEFAULT NULL ,parent_id
bigint(20) DEFAULT NULL ,parent_code
varchar(50) DEFAULT NULL ,grid_classification
varchar(10) NOT NULL ,grid_level
varchar(10) NOT NULL ,grid_property
varchar(10) DEFAULT NULL ',is_end
char(1) NOT NULL ,grid_address
varchar(200) DEFAULT NULL ,grid_introduce
varchar(1024) DEFAULT NULL ,) ENGINE=OLAP
UNIQUE KEY(
grid_id
)COMMENT "OLAP"
DISTRIBUTED BY HASH(
grid_id
) BUCKETS 3PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"in_memory" = "false",
"storage_format" = "V2"
);
sync jobs
CREATE SYNC social_system.sync_bm_grid
(
FROM social_sys.bm_grid INTO bm_grid_dor
)
FROM BINLOG
(
"type" = "canal",
"canal.server.ip" = "172.16.10.36",
"canal.server.port" = "11111",
"canal.destination" = "example",
"canal.username" = "canal",
"canal.password" = "canal"
);
canal instance.properties
canal.instance.mysql.slaveId=12341
canal.instance.gtidon=false
canal.instance.master.address=172.16.10.106:31016
canal.instance.master.journal.name=
canal.instance.master.position=
canal.instance.master.timestamp=
canal.instance.master.gtid=
canal.instance.rds.accesskey=
canal.instance.rds.secretkey=
canal.instance.rds.instanceId=
canal.instance.tsdb.enable=true
canal.instance.dbUsername=system
canal.instance.dbPassword=CCYdPAWF%Na8^roa
canal.instance.connectionCharset = UTF-8
canal.instance.defaultDatabaseName=social_sys
canal.instance.enableDruid=false
canal.instance.filter.regex=.\..
canal.instance.filter.black.regex=mysql\.slave_.*
canal.mq.topic=bm_mark_topic
canal.mq.partition=0
#################################################
What You Expected?
实现使用binlog load方式同步数据
How to Reproduce?
Java8编译,
系统版本:Linux localhost.localdomain 3.10.0-1062.el7.x86_64 #1 SMP Wed Aug 7 18:08:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
canal:0.
Anything Else?
主库手动增加3条数据,fe 错误日志:
2022-02-21 17:05:21,160 WARN (thrift-server-pool-12|237) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 3: errCode = 2, detailMessage = transaction not found
2022-02-21 17:06:03,062 WARN (thrift-server-pool-13|241) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 4: errCode = 2, detailMessage = transaction not found
2022-02-21 17:06:13,548 WARN (thrift-server-pool-11|193) [FrontendServiceImpl.loadTxnRollback():888] failed to rollback txn 5: errCode = 2, detailMessage = transaction not found
Are you willing to submit PR?
Code of Conduct
The text was updated successfully, but these errors were encountered: