一、启动报错
datax 连接mysql报错,datax No suitable driver found for ["jdbc:mysql:
报的驱动错误,如果已经安装了mysql相关的插件,一般是配置文件错误造成的。
如:(注意 reader 和 writer 的 jdbcUrl ,前者是个数组,后者是个字符串。本人把后者配置也成[] 报了上面的错误
{
"job": {
"setting": {
"speed": {
"channel": 1
},
"errorLimit": {
"record": 0,
"percentage": 0
}
},
"content": [
{
"reader": {
"name": "mysqlreader",
"parameter": {
"username": "root",
"password": "qwer1234",
"column": ["*"],
"connection": [
{
"jdbcUrl": ["jdbc:mysql://192.168.88.114:3306/dpcsys_test"],
"table": ["dpc_his_notice_detailed_tab"]
}
]
}
},
"writer": {
"name": "mysqlwriter",
"parameter": {
"write": {
"check": false # 设置忽略脏数据检查
},
"writeMode": "update",
"column": ["*"],
"session": [
"set session sql_mode='ANSI'"
],
"connection": [
{
"jdbcUrl": "jdbc:mysql://127.0.0.1:3306/dpcsys_test",
"table": ["dpc_his_notice_detailed_testmonth"]
}
],
"username": "root",
"password": "qwer1234"
}
}
}
]
}
}
二、同步过程中报错
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Framework-14], Description:[DataX传输脏数据超过用户预期,该错误通常是由于源端数据存在较多业务脏数据导致,请仔细检查DataX汇报的脏数据日志信息, 或者您可以适当调大脏数据阈值 .]. - 脏数据条数检查不通过,限制是[0]条,但实际上捕获了[14]条.
ERROR StdoutPluginCollector - 脏数据:
{"exception":"Table has no partition for value 740559","record":[{"byteSize":19,"index":0,"rawData":1172967129725534208,"type":"LONG"},{"byteSize":10,"index":1,"rawData":"1817127676","type":"STRING"},{"byteSize":2,"index":2,"rawData":"AI","type":"STRING"},{"byteSize":1,"index":3,"rawData":"0","type":"STRING"},{"byteSize":4,"index":4,"rawData":3897,"type":"LONG"},{"byteSize":1,"index":5,"rawData":8,"type":"LONG"},{"byteSize":1,"index":6,"rawData":"1","type":"STRING"},{"byteSize":5,"index":7,"rawData":18031,"type":"LONG"},{"byteSize":4,"index":8,"rawData":"3958","type":"STRING"},{"byteSize":5,"index":9,"rawData":26477,"type":"LONG"},{"byteSize":1,"index":10,"rawData":"1","type":"STRING"},{"byteSize":7,"index":11,"rawData":1869542,"type":"LONG"},{"byteSize":1,"index":12,"rawData":"0","type":"STRING"},{"byteSize":3,"index":13,"rawData":"369","type":"STRING"},{"byteSize":4,"index":14,"rawData":"环境湿度","type":"STRING"},{"byteSize":21,"index":15,"rawData":"DHS_13652259383777545","type":"STRING"},{"byteSize":1,"index":16,"rawData":"2","type":"STRING"},{"byteSize":3,"index":17,"rawData":"157","type":"STRING"},{"byteSize":5,"index":18,"rawData":"81.79","type":"STRING"},{"byteSize":0,"index":19,"rawData":"","type":"STRING"},{"byteSize":0,"index":20,"rawData":"","type":"STRING"},{"byteSize":8,"index":21,"rawData":1817127675000,"type":"DATE"},{"byteSize":0,"index":22,"type":"DATE"},{"byteSize":0,"index":23,"type":"STRING"},{"byteSize":8,"index":24,"rawData":1690898404000,"type":"DATE"},{"byteSize":8,"index":25,"rawData":1701303827000,"type":"DATE"},{"byteSize":0,"index":26,"type":"STRING"},{"byteSize":0,"index":27,"type":"STRING"},{"byteSize":5,"index":28,"rawData":"81.99","type":"STRING"},{"byteSize":0,"index":29,"type":"LONG"},{"byteSize":0,"index":30,"type":"STRING"},{"byteSize":12,"index":31,"rawData":"QY1050000003","type":"STRING"},{"byteSize":1,"index":32,"rawData":"0","type":"STRING"},{"byteSize":0,"index":33,"type":"STRING"},{"byteSize":8,"index":34,"rawData":1699698760000,"type":"DATE"},{"byteSize":11,"index":35,"rawData":"BengalAdmin","type":"STRING"},{"byteSize":8,"index":36,"rawData":1701303828000,"type":"DATE"},{"byteSize":0,"index":37,"type":"STRING"},{"byteSize":0,"index":38,"type":"STRING"},{"byteSize":0,"index":39,"type":"STRING"},{"byteSize":0,"index":40,"type":"STRING"},{"byteSize":0,"index":41,"type":"STRING"},{"byteSize":0,"index":42,"type":"LONG"}],"type":"writer"}
看日志发现 "type":"LONG" 可能数据类型Long有关?
发表评论