一、启动报错
datax 连接mysql报错,datax No suitable driver found for ["jdbc:mysql:
报的驱动错误,如果已经安装了mysql相关的插件,一般是配置文件错误造成的。
如:(注意 reader 和 writer 的 jdbcUrl ,前者是个数组,后者是个字符串。本人把后者配置也成[] 报了上面的错误
{ "job": { "setting": { "speed": { "channel": 1 }, "errorLimit": { "record": 0, "percentage": 0 } }, "content": [ { "reader": { "name": "mysqlreader", "parameter": { "username": "root", "password": "qwer1234", "column": ["*"], "connection": [ { "jdbcUrl": ["jdbc:mysql://192.168.88.114:3306/dpcsys_test"], "table": ["dpc_his_notice_detailed_tab"] } ] } }, "writer": { "name": "mysqlwriter", "parameter": { "write": { "check": false # 设置忽略脏数据检查 }, "writeMode": "update", "column": ["*"], "session": [ "set session sql_mode='ANSI'" ], "connection": [ { "jdbcUrl": "jdbc:mysql://127.0.0.1:3306/dpcsys_test", "table": ["dpc_his_notice_detailed_testmonth"] } ], "username": "root", "password": "qwer1234" } } } ] } }
二、同步过程中报错
经DataX智能分析,该任务最可能的错误原因是:
com.alibaba.datax.common.exception.DataXException: Code:[Framework-14], Description:[DataX传输脏数据超过用户预期,该错误通常是由于源端数据存在较多业务脏数据导致,请仔细检查DataX汇报的脏数据日志信息, 或者您可以适当调大脏数据阈值 .]. - 脏数据条数检查不通过,限制是[0]条,但实际上捕获了[14]条.
ERROR StdoutPluginCollector - 脏数据:
{"exception":"Table has no partition for value 740559","record":[{"byteSize":19,"index":0,"rawData":1172967129725534208,"type":"LONG"},{"byteSize":10,"index":1,"rawData":"1817127676","type":"STRING"},{"byteSize":2,"index":2,"rawData":"AI","type":"STRING"},{"byteSize":1,"index":3,"rawData":"0","type":"STRING"},{"byteSize":4,"index":4,"rawData":3897,"type":"LONG"},{"byteSize":1,"index":5,"rawData":8,"type":"LONG"},{"byteSize":1,"index":6,"rawData":"1","type":"STRING"},{"byteSize":5,"index":7,"rawData":18031,"type":"LONG"},{"byteSize":4,"index":8,"rawData":"3958","type":"STRING"},{"byteSize":5,"index":9,"rawData":26477,"type":"LONG"},{"byteSize":1,"index":10,"rawData":"1","type":"STRING"},{"byteSize":7,"index":11,"rawData":1869542,"type":"LONG"},{"byteSize":1,"index":12,"rawData":"0","type":"STRING"},{"byteSize":3,"index":13,"rawData":"369","type":"STRING"},{"byteSize":4,"index":14,"rawData":"环境湿度","type":"STRING"},{"byteSize":21,"index":15,"rawData":"DHS_13652259383777545","type":"STRING"},{"byteSize":1,"index":16,"rawData":"2","type":"STRING"},{"byteSize":3,"index":17,"rawData":"157","type":"STRING"},{"byteSize":5,"index":18,"rawData":"81.79","type":"STRING"},{"byteSize":0,"index":19,"rawData":"","type":"STRING"},{"byteSize":0,"index":20,"rawData":"","type":"STRING"},{"byteSize":8,"index":21,"rawData":1817127675000,"type":"DATE"},{"byteSize":0,"index":22,"type":"DATE"},{"byteSize":0,"index":23,"type":"STRING"},{"byteSize":8,"index":24,"rawData":1690898404000,"type":"DATE"},{"byteSize":8,"index":25,"rawData":1701303827000,"type":"DATE"},{"byteSize":0,"index":26,"type":"STRING"},{"byteSize":0,"index":27,"type":"STRING"},{"byteSize":5,"index":28,"rawData":"81.99","type":"STRING"},{"byteSize":0,"index":29,"type":"LONG"},{"byteSize":0,"index":30,"type":"STRING"},{"byteSize":12,"index":31,"rawData":"QY1050000003","type":"STRING"},{"byteSize":1,"index":32,"rawData":"0","type":"STRING"},{"byteSize":0,"index":33,"type":"STRING"},{"byteSize":8,"index":34,"rawData":1699698760000,"type":"DATE"},{"byteSize":11,"index":35,"rawData":"BengalAdmin","type":"STRING"},{"byteSize":8,"index":36,"rawData":1701303828000,"type":"DATE"},{"byteSize":0,"index":37,"type":"STRING"},{"byteSize":0,"index":38,"type":"STRING"},{"byteSize":0,"index":39,"type":"STRING"},{"byteSize":0,"index":40,"type":"STRING"},{"byteSize":0,"index":41,"type":"STRING"},{"byteSize":0,"index":42,"type":"LONG"}],"type":"writer"}
看日志发现 "type":"LONG" 可能数据类型Long有关?
发表评论