[Bug] IoTDB v1.3.0 版本不定时出现重启之后，结构和数据均查询不到的情况

Question

[Bug] IoTDB v1.3.0 版本不定时出现重启之后，结构和数据均查询不到的情况

SunnySaman opened this issue 8 months ago · comments

SunnySaman commented 8 months ago

Search before asking

I searched in the issues and found nothing similar.

Version

v1.3.0

Describe the bug and provide the minimal reproduce step

v1.3.0版本
部署环境：

Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.6 LTS
Release:	20.04
Codename:	foca

JDK：

openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)

安装方式：Standalone 安装

修改的配置如下：

common修改

config_node_consensus_protocol_class=org.apache.iotdb.consensus.simple.SimpleConsensus
schema_replication_factor=1
schema_region_consensus_protocol_class=org.apache.iotdb.consensus.simple.SimpleConsensus
data_replication_factor=1
data_region_consensus_protocol_class=org.apache.iotdb.consensus.simple.SimpleConsensus

confignode修改：

cn_internal_address=10.166.50.30
cn_seed_config_node=10.166.50.30:10710

datanode 修改：

dn_internal_address=10.166.50.30
dn_seed_config_node=10.166.50.30:10710

启动方式

先启动 confignode

  [Unit]
  Description=iotdb-confignode
  Documentation=https://iotdb.apache.org/
  After=network.target

  [Service]
  StandardOutput=null
  StandardError=null
  LimitNOFILE=65536
  Type=simple
  User=root
  Group=root
  Environment=JAVA_HOME=/usr/local/jdk-11.0.2
  ExecStart=/usr/local/apache-iotdb-1.3.0-all-bin/sbin/start-confignode.sh
  ExecStop=/usr/local/apache-iotdb-1.3.0-all-bin/sbin/stop-confignode.sh
  Restart=on-failure
  SuccessExitStatus=143
  RestartSec=5
  StartLimitInterval=600s
  StartLimitBurst=3
  RestartPreventExitStatus=SIGKILL

  [Install]
  WantedBy=multi-user.target

后启动 datanode

  [Unit]
  Description=iotdb-datanode
  Documentation=https://iotdb.apache.org/
  After=network.target

  [Service]
  StandardOutput=null
  StandardError=null
  LimitNOFILE=65536
  Type=simple
  User=root
  Group=root
  Environment=JAVA_HOME=/usr/local/jdk-11.0.2
  ExecStart=/usr/local/apache-iotdb-1.3.0-all-bin/sbin/start-datanode.sh
  ExecStop=/usr/local/apache-iotdb-1.3.0-all-bin/sbin/stop-datanode.sh
  Restart=on-failure
  SuccessExitStatus=143
  RestartSec=5
  StartLimitInterval=600s
  StartLimitBurst=3
  RestartPreventExitStatus=SIGKILL

  [Install]
  WantedBy=multi-user.target

写入数据方式：

通过JDBC写入

public class JDBCExample2 {
private static Logger logger = LoggerFactory.getLogger(JDBCExample.class);
public static String GetPVPath() {
    Random random = new Random();
    int sId = random.nextInt(6) + 1;
    int sysId = random.nextInt(1000) + 1;
    StringBuilder pvIdBuilder = new StringBuilder();
    String characters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
    for (int i = 0; i < 10; i++) {
    int index = random.nextInt(characters.length());
    char ch = characters.charAt(index);
    pvIdBuilder.append(ch);
    }
    String pvId = pvIdBuilder.toString();

    String result = String.format("root.fz4.pv.s_%d.sys_%d.pv_%s", sId, sysId, pvId);
    String pvInsert =
            "insert into "+result+"(timestamp, is_alarm, is_bad, pv_name, subsystem_id, equipment_name, pv_type, io_type, server_time, pv_time, " +
                    "original, string_value, int_value, double_value, boolean_value, dword_value, long_value, float_value, char_value, word_value, station_id) " +
                    "aligned VALUES(NOW(),False,False,'ceshi',110,'ceshi',9,1,1,1,'0.3','',null,null,null,null,null,1.1,'ceshi',null,1)";
    System.out.println(pvInsert);
    return pvInsert;
}
public static void main(String[] args) throws ClassNotFoundException, SQLException {
    Class.forName("org.apache.iotdb.jdbc.IoTDBDriver");
    try{
    while (true){
        Connection connection =
                DriverManager.getConnection(
                        "jdbc:iotdb://10.166.50.30:6667?version=V_1_0", "root", "root");
        Statement statement = connection.createStatement();
        String sql = GetPVPath();
        for (int i = 1; i <= 100; i++) {
        statement.addBatch(sql);
        }
        statement.executeBatch();
        statement.clearBatch();
        statement.close();
        connection.close();
    }
    } catch (IoTDBSQLException e) {
    logger.error("IoTDB Jdbc example error", e);
    }
}

}

写入结果：均正常写入，并可以在线查询到结果
重启方式：../sbin/stop-standalone方式和systemctl stop iotdb-confignode 、systemctl stop iotdb-datanode
重启后，查询timeseries和数据，均查询不到。

再次写入：

通过shell端写入sql

CREATE DATABASE root.sg1;
CREATE TIMESERIES root.sg1.d1.s1 WITH DATATYPE=INT64, ENCODING=RLE, COMPRESSOR=SNAPPY;
CREATE TIMESERIES root.sg1.d1.s2 WITH DATATYPE=INT64, ENCODING=RLE, COMPRESSOR=SNAPPY;
CREATE TIMESERIES root.sg1.d1.s3 WITH DATATYPE=INT64, ENCODING=RLE, COMPRESSOR=SNAPPY;
insert into root.sg1.d1(timestamp, s1, s2, s3) values(now(),1,2,3);

通过JDBC写入（和上面JDBC写入方式一样）

重启
重启后，查询timeseries和数据，均可以正常查询到

What did you expect to see?

expect ：
JDBC写入重启可以正常查询到数据

What did you see instead?

怀疑JDBC写入有问题

Anything else?

1

Are you willing to submit a PR?

I'm willing to submit a PR!

Jackie Tien · Answer 1 · Thu Jan 25 2024 22:14:06 GMT+0800 (China Standard Time)

我用你的JDBC程序试了一下，并没有复现。是对于JDBC程序运行的时间有要求吗？我看你的JDBC程序是写了死循环一直写的。

Jackie Tien · Answer 2 · Thu Jan 25 2024 22:15:23 GMT+0800 (China Standard Time)

有额外的操作吗？建模板之类的。

Haonan · Answer 3 · Fri Jan 26 2024 11:18:58 GMT+0800 (China Standard Time)

有模版

有额外的操作吗？建模板之类的。

SunnySaman · Answer 4 · Fri Jan 26 2024 11:31:19 GMT+0800 (China Standard Time)

这是模板

create storage group root.fz4.pv.s_1;
create storage group root.fz4.pv.s_2;
create storage group root.fz4.pv.s_3;
create storage group root.fz4.pv.s_4;
create storage group root.fz4.pv.s_5;
create storage group root.fz4.pv.s_6;
create storage group root.fz4.pv.s_7;
create storage group root.fz4.pv.s_8;
create storage group root.fz4.pv.s_9;
create storage group root.fz4.pv.s_49;
create storage group root.fz4.pv.s_52;

create schema template t4pv aligned (is_alarm boolean encoding=RLE,is_bad boolean encoding=RLE,pv_name text encoding=DICTIONARY,subsystem_id int32 encoding=RLE,equipment_name text encoding=DICTIONARY,pv_type int32 encoding=RLE,io_type int32 encoding=RLE,server_time int64 encoding=RLE,pv_time int64 encoding=RLE,original text encoding=PLAIN,string_value text encoding=DICTIONARY,int_value int32 encoding=RLE,double_value double encoding=GORILLA,boolean_value boolean encoding=RLE,dword_value int64 encoding=RLE,long_value int64 encoding=RLE,station_id int32 encoding=RLE,float_value float encoding=RLE,char_value text encoding=PLAIN,word_value int32 encoding=RLE);

set schema template t4pv to root.fz4.pv.s_1;
set schema template t4pv to root.fz4.pv.s_2;
set schema template t4pv to root.fz4.pv.s_3;
set schema template t4pv to root.fz4.pv.s_4;
set schema template t4pv to root.fz4.pv.s_5;
set schema template t4pv to root.fz4.pv.s_6;
set schema template t4pv to root.fz4.pv.s_7;
set schema template t4pv to root.fz4.pv.s_8;
set schema template t4pv to root.fz4.pv.s_9;
set schema template t4pv to root.fz4.pv.s_49;
set schema template t4pv to root.fz4.pv.s_52;

Jackie Tien · Answer 5 · Fri Jan 26 2024 14:23:33 GMT+0800 (China Standard Time)

那是已知问题，不过应该show timeseries是能够查到元数据的，只是select查不到数据，我们会在1.3.1版本里进行修复，修复pr也已经提出，等合入后，也可以自行先打一个包。

SunnySaman · Answer 6 · Fri Jan 26 2024 14:27:53 GMT+0800 (China Standard Time)

show timeseries 模板序列、数据在重启后确实查不到