zhuyijun/nacos
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #484 from alibaba/feature_cmdb

Browse Source 
Feature cmdb
This commit is contained in:
Fury Zhu 2018-12-18 18:01:27 +08:00 committed by GitHub
commit 2bde25bc33
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
8 changed files with 133 additions and 56 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

108
CHANGELOG.md
View File

@ -1,7 +1,21 @@
## 0.7.0(Dec, 2018)
* [ #461 ] Registration failed when instance port is set to 0
* [ #455 ] The console can't change the change code
* [ #447 ] 集群模式server挂掉一台后，提供方注册失败
* [ #445 ] 0.6.1控制台创建配置发布提交时，提示信息有问题
* [ #442 ] Typos in class names and variables.
* [ #413 ] The console has some uncaught exceptions
* [ #395 ] nacos surport mysql in the case of stand-alone mode
* [ #393 ] Support operation of selector on console
* [ #365 ] NodeJs SDK support
* [ #362 ] The metadata will lost when online or offline instance through web ui
* [ #187 ] Provide Label ability for Naming Service into NACOS for complex multi-DC scenario.
## 0.6.1(Dec, 2018)
[#421] NamingService's serivce name can't use colon(:) in Windows
[#432] When packing nacos-core, ${user.home} is replaced in the logback configuration file (nacos.xml)
* [#421] NamingService's serivce name can't use colon(:) in Windows
* [#432] When packing nacos-core, ${user.home} is replaced in the logback configuration file (nacos.xml)
## 0.6.0(Dec, 2018)
@ -16,61 +30,61 @@
## 0.5.0(Nov, 2018)
[#148] Naming write performace.
[#175] Support deregistering instance automatically.
[#176] Naming client query instance method should bypass local cache at client start.
[#177] Console supports registering new empty service and delete empty service.
[#181] NPE when adding a instance if no leader in the raft cluster.
[#193] Configure host domain name cause nacos server cluster is unavailable.
[#209] Disable service and cluster level customization in client registerInstance method.
[#214] Please support Java 11.
[#222] print more nacos server start status info in start.log.
[#231] Refactoring: Parsing the Nacos home directory and the cluster.conf file.
[#246] "mvn -B clean apache-rat:check findbugs:findbugs" did not work as expected.
[#251] Console Editor Optimization.
[#254] DataId and group are required in historical version and listener query.
[#256] Whether the service discovery data needs to add a newline link symbol.
[#257] Listening query switching query dimension data is not refreshed.
[#258] Remove the Balloon of DataId/Group.
[#259] Listening query paging size problem.
[#272] "#it is ip" is also parsed into an instance IP.
[#275] nacos coredns plugin to support DNS.
[#281] We should lint the console code.
[#302] Maven build project supports java 11.
[#316] In stand alone mode, Nacos still checks the cluster.conf.
* [#148] Naming write performace.
* [#175] Support deregistering instance automatically.
* [#176] Naming client query instance method should bypass local cache at client start.
* [#177] Console supports registering new empty service and delete empty service.
* [#181] NPE when adding a instance if no leader in the raft cluster.
* [#193] Configure host domain name cause nacos server cluster is unavailable.
* [#209] Disable service and cluster level customization in client registerInstance method.
* [#214] Please support Java 11.
* [#222] print more nacos server start status info in start.log.
* [#231] Refactoring: Parsing the Nacos home directory and the cluster.conf file.
* [#246] "mvn -B clean apache-rat:check findbugs:findbugs" did not work as expected.
* [#251] Console Editor Optimization.
* [#254] DataId and group are required in historical version and listener query.
* [#256] Whether the service discovery data needs to add a newline link symbol.
* [#257] Listening query switching query dimension data is not refreshed.
* [#258] Remove the Balloon of DataId/Group.
* [#259] Listening query paging size problem.
* [#272] "#it is ip" is also parsed into an instance IP.
* [#275] nacos coredns plugin to support DNS.
* [#281] We should lint the console code.
* [#302] Maven build project supports java 11.
* [#316] In stand alone mode, Nacos still checks the cluster.conf.
## 0.4.0(Nov 7, 2018)
[#216] Fix tenant dir problem
[#197] Service update ignored some properties
[#190] Client beat lose weight info and metadata info
[#188] Console delete data cannot be updated in time
[#179] Listening query fail when namespace is not blank
[#157] Lack information in readme.md to describe the related project repositories for Nacos echosystem
[#144] There have a error and something are not clear
[#106] Snapshot file create error
[#92] Eliminate warnings, refactor code, show start.log detail
* [#216] Fix tenant dir problem
* [#197] Service update ignored some properties
* [#190] Client beat lose weight info and metadata info
* [#188] Console delete data cannot be updated in time
* [#179] Listening query fail when namespace is not blank
* [#157] Lack information in readme.md to describe the related project repositories for Nacos echosystem
* [#144] There have a error and something are not clear
* [#106] Snapshot file create error
* [#92] Eliminate warnings, refactor code, show start.log detail
## 0.3.0(Oct 26, 2018)
[#171] UI debug errors
[#156] Web UI 404 problem
[#155] use local resource
[#145] nacos-example not found :org.apache.logging.log4j.core.Logger
[#142] UI console show Group
[#149] Fix naming client beat process failed bug.
[#150] Fix naming service registration hangs bug.
* [#171] UI debug errors
* [#156] Web UI 404 problem
* [#155] use local resource
* [#145] nacos-example not found :org.apache.logging.log4j.core.Logger
* [#142] UI console show Group
* [#149] Fix naming client beat process failed bug.
* [#150] Fix naming service registration hangs bug.
## 0.3.0-RC1(Oct 19, 2018)
[#33] Support console for config management.
[#51] Support console for naming service.
[#121] Fix get instance method hanging bug.
[#138] Add a flag to indicate if instance is offline.
[#130] Fix health check disabled if machine has one CPU core bug.
[#139] Fix still get instance with zero weight bug.
[#128] Fix console layout bug.
* [#33] Support console for config management.
* [#51] Support console for naming service.
* [#121] Fix get instance method hanging bug.
* [#138] Add a flag to indicate if instance is offline.
* [#130] Fix health check disabled if machine has one CPU core bug.
* [#139] Fix still get instance with zero weight bug.
* [#128] Fix console layout bug.

8
api/src/main/java/com/alibaba/nacos/api/cmdb/spi/CmdbService.java
View File

@ -55,21 +55,21 @@ public interface CmdbService {
/**
* Get label value of label name of ip
*
* @param entityName entity name
* @param entityType entity type
* @param entityValue entity value
* @param labelName target label name
* @return label value
*/
String getLabelValue(String entityValue, String entityType, String labelName);
String getLabelValue(String entityName, String entityType, String labelName);
/**
* Get all label value of ip
*
* @param entityName entity name
* @param entityType entity type
* @param entityValue entity value
* @return all label values
*/
Map<String, String> getLabelValues(String entityValue, String entityType);
Map<String, String> getLabelValues(String entityName, String entityType);
/**
* Dump all entities in CMDB

4
client/src/test/java/com/alibaba/nacos/client/NamingTest.java
View File

@ -42,11 +42,11 @@ public class NamingTest {
instance.setPort(80);
instance.setWeight(2);
Map<String, String> map = new HashMap<String, String>();
map.put("env", "prod");
map.put("netType", "external");
map.put("version", "2.0");
instance.setMetadata(map);
namingService.registerInstance("dungu.test.1", instance);
namingService.registerInstance("nacos.test.1", instance);
ExpressionSelector expressionSelector = new ExpressionSelector();
expressionSelector.setExpression("INSTANCE.metadata.registerSource = 'dubbo'");

2
cmdb/src/main/java/com/alibaba/nacos/cmdb/controllers/OperationController.java
View File

@ -55,6 +55,8 @@ public class OperationController {
case "loadDataAtStart":
switches.setLoadDataAtStart(Boolean.parseBoolean(value));
break;
case "labelTaskInterval":
switches.setLabelTaskInterval(Integer.parseInt(value));
default:
break;
}

11
cmdb/src/main/java/com/alibaba/nacos/cmdb/core/SwitchAndOptions.java
View File

@ -30,6 +30,9 @@ public class SwitchAndOptions {
@Value("${nacos.cmdb.eventTaskInterval}")
private int eventTaskInterval;
@Value("${nacos.cmdb.labelTaskInterval}")
private int labelTaskInterval;
@Value("${nacos.cmdb.loadDataAtStart}")
private boolean loadDataAtStart;
@ -49,6 +52,14 @@ public class SwitchAndOptions {
this.eventTaskInterval = eventTaskInterval;
}
public int getLabelTaskInterval() {
return labelTaskInterval;
}
public void setLabelTaskInterval(int labelTaskInterval) {
this.labelTaskInterval = labelTaskInterval;
}
public boolean isLoadDataAtStart() {
return loadDataAtStart;
}

54
cmdb/src/main/java/com/alibaba/nacos/cmdb/memory/CmdbProvider.java
View File

@ -15,6 +15,7 @@
*/
package com.alibaba.nacos.cmdb.memory;
import com.alibaba.fastjson.JSON;
import com.alibaba.nacos.api.cmdb.spi.CmdbService;
import com.alibaba.nacos.api.cmdb.pojo.Entity;
import com.alibaba.nacos.api.cmdb.pojo.EntityEvent;
@ -76,8 +77,6 @@ public class CmdbProvider implements CmdbReader, CmdbWriter {
return;
}
// TODO load data on disk:
// init label map:
Set<String> labelNames = cmdbService.getLabelNames();
if (labelNames == null || labelNames.isEmpty()) {
@ -103,6 +102,7 @@ public class CmdbProvider implements CmdbReader, CmdbWriter {
load();
UtilsAndCommons.GLOBAL_EXECUTOR.schedule(new CmdbDumpTask(), switches.getDumpTaskInterval(), TimeUnit.SECONDS);
UtilsAndCommons.GLOBAL_EXECUTOR.schedule(new CmdbLabelTask(), switches.getLabelTaskInterval(), TimeUnit.SECONDS);
UtilsAndCommons.GLOBAL_EXECUTOR.schedule(new CmdbEventTask(), switches.getEventTaskInterval(), TimeUnit.SECONDS);
}
@ -142,19 +142,61 @@ public class CmdbProvider implements CmdbReader, CmdbWriter {
entityMap.get(entity.getType()).put(entity.getName(), entity);
}
public class CmdbLabelTask implements Runnable {
@Override
public void run() {
Loggers.MAIN.debug("LABEL-TASK {}", "start dump.");
if (cmdbService == null) {
return;
}
try {
Map<String, Label> tmpLabelMap = new HashMap<>(16);
Set<String> labelNames = cmdbService.getLabelNames();
if (labelNames == null || labelNames.isEmpty()) {
Loggers.MAIN.warn("CMDB-LABEL-TASK {}", "load label names failed!");
} else {
for (String labelName : labelNames) {
// If get null label, it's still ok. We will try it later when we meet this label:
tmpLabelMap.put(labelName, cmdbService.getLabel(labelName));
}
if (Loggers.MAIN.isDebugEnabled()) {
Loggers.MAIN.debug("LABEL-TASK {}", "got label map:" + JSON.toJSONString(tmpLabelMap));
}
labelMap = tmpLabelMap;
}
} catch (Exception e) {
Loggers.MAIN.error("CMDB-LABEL-TASK {}", "dump failed!", e);
} finally {
UtilsAndCommons.GLOBAL_EXECUTOR.schedule(this, switches.getLabelTaskInterval(), TimeUnit.SECONDS);
}
}
}
public class CmdbDumpTask implements Runnable {
@Override
public void run() {
try {
Loggers.MAIN.debug("DUMP-TASK {}", "start dump.");
if (cmdbService == null) {
return;
}
// refresh entity map:
entityMap = cmdbService.getAllEntities();
} catch (Exception e) {
Loggers.MAIN.error("CMDB-DUMP {}", "dump failed!", e);
Loggers.MAIN.error("DUMP-TASK {}", "dump failed!", e);
} finally {
UtilsAndCommons.GLOBAL_EXECUTOR.schedule(this, switches.getDumpTaskInterval(), TimeUnit.SECONDS);
}
@ -167,6 +209,8 @@ public class CmdbProvider implements CmdbReader, CmdbWriter {
public void run() {
try {
Loggers.MAIN.debug("EVENT-TASK {}", "start dump.");
if (cmdbService == null) {
return;
}
@ -175,6 +219,10 @@ public class CmdbProvider implements CmdbReader, CmdbWriter {
List<EntityEvent> events = cmdbService.getEntityEvents(eventTimestamp);
eventTimestamp = current;
if (Loggers.MAIN.isDebugEnabled()) {
Loggers.MAIN.debug("EVENT-TASK {}", "got events size:" + ", events:" + JSON.toJSONString(events));
}
if (events != null && !events.isEmpty()) {
for (EntityEvent event : events) {

1
console/src/main/resources/META-INF/nacos-default.properties
View File

@ -32,6 +32,7 @@ OPEN_SPAS=true
nacos.cmdb.dumpTaskInterval=3600
nacos.cmdb.eventTaskInterval=10
nacos.cmdb.labelTaskInterval=300
nacos.cmdb.loadDataAtStart=false
db.num=2

1
distribution/conf/application.properties
View File

@ -6,6 +6,7 @@ server.port=8848
nacos.cmdb.dumpTaskInterval=3600
nacos.cmdb.eventTaskInterval=10
nacos.cmdb.labelTaskInterval=300
nacos.cmdb.loadDataAtStart=false
#db.num=2