阿里云 AnalyticDB MySQL 数据库,批量插入 返回

SqlSugar 沟通中
13 252
该叫什么 Leonid 发布于2周前
悬赏:0 飞吻

数据插入,大概8000+数据


1、 await ljjDataAdb.Insertable(refundDetailNewEntities).ExecuteCommandAsync(); 


返回  Failed to read the result set.


2、await ljjDataAdb.Insertable(refundDetailNewEntities).PageSize(1000).ExecuteCommandAsync();


返回 Connection must be Open; current state is Closed


3、await ljjDataAdb.Fastest<RefundDetailNewEntity>().PageSize(3000).BulkCopyAsync(refundDetailNewEntities);


返回 Syntax error,llegal charset pos 118, line 1.column 112, token IDENTIFER utfemb4


连接串增加了AllowLoadLocalInfile=true;Charset=utf8mb4;


使用以下语句可以插入,但是插入效率不太满意,大概需要10秒左右。

await ljjDataAdb.Insertable(refundDetailNewEntities).UseParameter().ExecuteCommandAsync();

Fastest模式该如何排查问题呢?


热忱回答13

  • 2、await ljjDataAdb.CopyNew().Insertable(refundDetailNewEntities).PageSize(1000).ExecuteCommandAsync();


    并且升级最新sqlsugar

    0 回复
  • @fate sta

    1、已将版本更新至 5.1.4.211

    2、await ljjDataAdb.CopyNew().Insertable(refundDetailNewEntities).PageSize(1000).ExecuteCommandAsync(); 依然报Connection must be Open; current state is Closed

    3、那怕第二点不报错,因在脚本里面加了select @@IDENTITY,最终也会提示Failed to read the result set。因为@@IDENTITY数据库不支持。用了PageSize(1000),就不能用UseParameter()

    0 回复
  • await ljjDataAdb.CopyNew().Insertable(refundDetailNewEntities).PageSize(1000).ExecuteCommandAsync(); 

    设置自动释放后不可能报这个错的。 Connection must be Open; current state is Closed

     可以单独写DEMO测试。

    0 回复
  • 或者是别的地方引起的。

    0 回复
  • @fate sta

    [HttpPost]
    public async Task<(string code, string message)> Test()
    {
        try
        {
            var ljjDataAdb = DbScoped.SugarScope.GetConnectionScope("LjjDataAdb");
            
            //或 DbScoped.SugarScope.GetConnection("LjjDataAdb");
            
            var refundTaskNewEntity = new RefundTaskNewEntity
            {
                Id = Guid.NewGuid().ToString("N"),
                TaskType = 2,
                OssUrl = "https://1111111.xlsx"
            };
            
            var basePath = Path.Combine(AppContext.BaseDirectory, refundTaskNewEntity.Id);
    
            if (Directory.Exists(basePath))
            {
                Directory.Delete(basePath, true);
            }
    
            Directory.CreateDirectory(basePath);
    
            var savePath = Path.Combine(basePath, $"{refundTaskNewEntity.Id}.xlsx");
    
            await DownloadFileAsync(refundTaskNewEntity.OssUrl, savePath, CancellationToken.None);
    
            var rows = (await MiniExcel.QueryAsync<AnalysisFileModel>(savePath)).ToList();
    
            var refundDetailNewEntities = new List<RefundDetailNewEntity>();
    
            for (var i = 0; i < rows.Count; i++)
            {
                var row = rows[i];
    
                var refundDetailNewEntity = new RefundDetailNewEntity
                {
                    Id = Guid.NewGuid().ToString("N"),
                    TaskId = refundTaskNewEntity.Id,
                    Status = 1,
                    PlatformOrderId = row.平台订单号?.Replace("'", "").Replace("`", ""),
                    RefundAmount = row.退款金额
                };
    
                refundDetailNewEntities.Add(refundDetailNewEntity);
            }
            
            await ljjDataAdb.CopyNew().Insertable(refundDetailNewEntities).PageSize(3000).ExecuteCommandAsync();
    
            return ("10000", "操作成功");
        }
        catch (Exception ex)
        {
            _logger.LogInformation(ex.Message);
            _logger.LogInformation(ex.StackTrace);
        }
    
        return ("99999", "系统异常");
    }

    image.png

    0 回复
  • @fate sta:麻烦帮忙看看

    0 回复
  • 当前代码看不出问题,检查是否是其他地方引起的。比如AOP或者中间件等。

    还有疑问提供可以重现的DEMO


    https://www.donet5.com/Home/Doc?typeId=2366 (提问模版)

    0 回复
  • @fate sta


    我可以提供相关用例及DEMO,但是数据库无法对外访问。这块怎么处理呢?

    0 回复
  • 提问模版会建表 建测试数据。你根本不需要提供库

    0 回复
  • 按模版提供DEMO

    0 回复
  • @fate sta:整个项目实际上是MySQL + AnalyticDB两种数据库。操作MySQL的时候没有任何问题。只有操作AnalyticDB才会有问题。所以如果您这边没有AnalyticDB数据库的环境,根本没办法复现问题的。

    0 回复
  • @fate sta:~~~

    0 回复
  • @fate sta


    AnalyticDBMySQL版本,调用 

    await db.Fastest<RefundDetailNewEntity>().BulkCopyAsync(list);
    
    提示返回 Syntax error,llegal charset pos 118, line 1.column 112, token IDENTIFER utfemb4


    因为在创建MySqlBulkLoader时,对CharacterSet进行了赋值。该数据库的LOAD DATA 功能不支持该属性,取消赋值即可。




    0 回复