From 8a54b1fc4cc750deccfdd9c0d95261f1da39ec45 Mon Sep 17 00:00:00 2001 From: liaochuntao Date: Wed, 14 Oct 2020 19:33:33 +0800 Subject: [PATCH] Naming module Raft protocol migration (#3989) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix-#3595, delete the unnecessary code (#3596) * [ISSUE #3566] move the permission code of nacos-core module to nacos-auth module (#3593) * move the permission code of nacos-core module to nacos-auth module. * Fix some code style issues * address server module auth package name change. * test change * Incorrect package name correction * [ISSUE #3592] Fix incorrect prompt when accessing the restricted namespace (#3603) * Fix incorrect prompt when accessing the restricted namespace * Modify variable name * [ISSUE #3600] Replace the deprecated api of jwt (#3616) * replace the deprecated api of jwt * transfer secretKey to byte array just using String encode with utf-8 * [ISSUE #3613] Fix `unit test method not be static` & update publish config listener in `ConfigTest.java` (#3614) * fix `unit test method not be static` & update publish config listener in `ConfigTest.java` * fix `unit test method not be static` & update publish config listener in `ConfigTest.java` * move jwt dependency from console,core to auth. (#3624) * refactor: unified implementation of http client api adjustment. (#3639) * [ISSUE #3628] set naming client updateTask interval more flexible (#3637) * 1.use server cacheMillis event service deleted 2.naming client UpdateTask's interval will inc by failCount that connect with server * 1.move failCount to updateTask 2.redefine the updateService method name. updateServiceNow -> updateService, wrap updateService in updateServiceNow when first getServiceInfo * 1.create push client even service is not exist 2.serviceInfo's hosts is empty or can't connect to server both add the updateTsk interval * format the indent * fix: create kvstorage * refactor: create kv storage] * refactor: refactor rocksdb storage code * Fix issue 3661 (#3662) * improvement: merge upstream/develop * feat: merge upstream develop * fix: fix issue #3661 * [ISSUE#3658] refactor TaskManager and move to nacos-common module (#3663) * Add TaskManagerTest * Move Abstract Task and Task processor to nacos-common * Add Nacos execute engine interface and nacos task interface * Refactor Task Manager to NacosDelayTaskExecuteEngine * for code style * [ISSUE #3671] move some tools class into common package (#3672) * for #3621 (#3668) * [ISSUE#3315]Nacos client support https (#3654) * [ISSUE #3315] nacos client support https * common module add tls related classes * JdkHttpClientRequest support https * unified IpUtils * [ISSUE #3315] nacos client support https * common module add tls related classes * JdkHttpClientRequest support https * unified IpUtils * [ISSUE #3315] nacos client support https * format code * fix typo and doc format of README file (#3688) 1. It’s a little weird to use Chinese comma in English 2. Fix typo: change 'reposity' to 'repository', 'instance' to 'instances' 3. Format other details of the doc * Try to fix nacos server CLOSE_WAIT (#3703) * Fix revert chunk isn't work in Content Comparison page (#3686) * fix: fixed cluster node version issue * Fix Logging in with the wrong username or password will cause jackson serialization results to fail (#3695) (#3721) * [ISSUE#3712] add apache http client factory (#3716) * refactor: Add apache http client Factory. * refactor: Add apache http client Factory. * add license * refactor: class name change * [Issue#3692] Use new distro task engine replace old task dispatcher. (#3715) * Add ServiceManagerTest * Add DistroConsistencyServiceImplTest * Add new distro entities * Add new distro sync data change * add new VerifyTask to do checksum. * Replace old sync task and checksum task * Add retry sync change task. * Fix high cpu load * For checkstyle * Combine naming sync task to reduce http cost * Move some classes * Refactor keys combined logic * enhance package construct * Fix unit test * For pmd * [Issue #3692] Use new distro implmentation to handle sync datum and checksum request (#3734) * Add receive data and verify data distro protocol interface * Use new distro implmentation to handle sync datum and checksum request * Use new distro implmentation to handle get datum request. * refactor: adjust the data loading logic when a new protocol is started * [ISSUE#3692] Use new distro implmentation to handle init all datum request. (#3744) * Add load data task in DistroProtocol * Use new distro implmentation to handle init all datum request. * Fix unit test * fix: #3617 (#3678) * fix #3617 * 调整代码格式 * 修改抛出的Exception类型 * 调整逻辑 * 移除没用到的方法 * 修改注释 * [ISSUE#3658] Move distro sync code to nacos-core module (#3750) * Move distro sync code to nacos-core module * Update unit test * feat: none * fix: fixing serialization problems * replace the deprecated api com.fasterxml.jackson.databind.node.ObjectNode#put(java.lang.String, com.fasterxml.jackson.databind.JsonNode) * Revert "[#3368]Cancel empty Long polling thread to improve performance. (#ISSUE3432)" (#3778) This reverts commit 95c8bf242b1f8895b77c519fb7dfdbc144e986c0. * [ISSUE #3658] Some enhance refactor for naming distro (#3765) * Some enhance refactor for naming distro * Remove null code * [ISSUE #3687] check serviceName's format(groupName@@serviceName) in server and client (#3767) * 1.in server, check serviceName's format 'groupName@@serviceName', groupName and serviceName can't be blank 2.in client, check 'groupName@@serviceName', groupName and serviceName can't be blank * ignore the check to groupName * check split's length instead of exception to check argument * 1.add some notes 2.remove unnecessary code * modify the notes * [ISSUE#3790] Supplement http response Content-Encoding processing. (#3791) * bug: fix issue #3790, Supplement http response Content-Encoding processing * bug: fix issue #3790, Supplement http response Content-Encoding processing. * bug: fix issue #3790, Supplement http response Content-Encoding processing. * Update ConvertUtils.java (#3789) * fix ConvertUtils can not handle FormatException. * fix: fixed some logic errors * [ISSUE#3192] naming module replace http client (#3763) * naming module replace http client * refactor: naming module replace http client. * refactor: naming module replace http client. * refactor: Add apache http client Factory. * refactor: naming module replace http client. * fix code style * refactor: Add http client config * refactor: naming module HttpClientManager change * refactor: naming module HttpClientManager change * refactor: naming module replace http client. * fix code style * refactor: fix JDK http client Use error problem. * refactor: Query And Header entity init Add non-empty judgment * Enhance the asynchronous http delete request method to support body passing parameters. * refactor: apache http client set MaxConnTotal and maxConnPerRoute. * Fix NullPointerException when no subscriber for slow event (#3835) * Set mediaType charset as utf8 (#3837) * Fix code style problem in DiskUtils (#3842) * Fix http client contentType charset problem (#3848) * feature issue #3804 (#3805) * improvement: merge upstream/develop * feat: merge upstream develop * feat: manage the loading of configuration files uniformly * fix: fix copyright style * style: fix code style * fix: fix code style * bug: fix the problem of incorrect judgment of http response code in SubscribeManager#getSubscribers() method (#3879) * [ISSUE #3867] replace the way which get version (#3872) * replace the way which get version * remove version sign application.properties in nacos-api * [ISSUE #3871] fix description don't match the error (#3886) * Fix jraft problem * Fix chinese string are truncated in ConcunrrentDiskUtil (#3883) * Temp fix raft server can't refresh raft configuration problem * update spring boot dependencies version (#3900) Co-authored-by: yanlinly Co-authored-by: 杨翊 SionYang <263976490@qq.com> * [ISSUE #3781]Fix service list intermittently lost service (#3891) * update service init * commit futureMap.remove() * update serviceManager Co-authored-by: yanlinly Co-authored-by: 杨翊 SionYang <263976490@qq.com> * Fix Listener do not listen new consistency problem * Fix Performance logger thread call old raft error * Use datum in new raft processor to compatible old data * Use multiple kv storage in new raft processor to compatible old data * Revert PR#2849. * Move datum key check to KeyBuilder * Fix 1.3.2 upgrade 1.4.0 can't notify service change problem * [ISSUE #3850] ignore socket exception when client destroy already (#3906) * if client destroy already, ignore socket exception. * remove the exception's judgement * remove unuseful import * fix: fix merge conflict * isAddressServerHealth set as true when request success (#3952) * [ISSUE#3533] change cache dir with namespace -- part 1: unify the cache dir (#3859) * [ISSUE#3533] change cache dir with namespace -- part 2: unify the log dir (#3882) * fix typo error (#3954) * [ISSUE #3909] add domain's judgement (#3913) * add domain's judgement * modify domain's judgement, can resolve = true * remove judgement in 'if' code block * replace Ip to IP in InetUtils * log warn info when domain can not be resolved * fix vaiable name * 1.fix unit test can't pass (#3956) 2.update the unit test * Move remove listener logic to ServiceManager * Move remove listener logic to ServiceManager * Fix#3973 (#3974) * fix #3973 * 重复代码抽取到一个方法 * 删除私有方法的注释 * 处理namespace参数的方法提出到一个工具类中 * 修改注释 * 添加licences * 增加 TenantUtil 的测试 * TenantUtil 改名为 NamespaceUtil * For #3384, Fix member extend info do not update error. (#3982) * refactor: code refactor Co-authored-by: 赵延 <1060026287@qq.com> Co-authored-by: mai.jh Co-authored-by: ljhrot <296093710@qq.com> Co-authored-by: Xarrow Co-authored-by: 杨翊 SionYang <263976490@qq.com> Co-authored-by: Gagharv Co-authored-by: syapollo <34880947+syapollo@users.noreply.github.com> Co-authored-by: S2W Co-authored-by: Tboy Co-authored-by: 邪影oO <213539@qq.com> Co-authored-by: yanlinly Co-authored-by: Mark4z <36187602+mark4z@users.noreply.github.com> Co-authored-by: Marcus Co-authored-by: shizhengxing <798187024@qq.com> Co-authored-by: ljhrot Co-authored-by: sanxun0325 Co-authored-by: JackSun-Developer --- .../nacos/api/naming/utils/NamingUtils.java | 3 + auth/pom.xml | 4 + .../nacos/auth/common/AuthConfigs.java | 10 +- .../auth/common/env/ReloadableConfigs.java | 75 --- .../client/config/impl/ClientWorker.java | 47 +- .../client/logging/AbstractNacosLogging.java | 6 +- .../client/naming/NacosNamingService.java | 11 +- .../naming/cache/ConcurrentDiskUtil.java | 5 +- .../client/naming/core/PushReceiver.java | 3 + .../alibaba/nacos/client/utils/ParamUtil.java | 16 +- .../src/main/resources/application.properties | 16 - client/src/main/resources/nacos-log4j2.xml | 8 +- client/src/main/resources/nacos-logback.xml | 8 +- .../listener/impl/ClientWorkerTest.java | 132 ---- .../client/naming/cache/DiskCacheTest.java | 10 +- .../http/AbstractApacheHttpClientFactory.java | 10 +- .../http/AbstractHttpClientFactory.java | 9 +- .../nacos/common/http/BaseHttpClient.java | 3 +- .../nacos/common/http/BaseHttpMethod.java | 36 ++ .../nacos/common/http/HttpClientConfig.java | 115 +++- .../alibaba/nacos/common/http/HttpUtils.java | 19 +- .../http/client/NacosAsyncRestTemplate.java | 21 + .../common/http/client/NacosRestTemplate.java | 20 + .../request/DefaultHttpClientRequest.java | 2 +- .../client/request/JdkHttpClientRequest.java | 2 +- .../response/JdkHttpClientResponse.java | 14 + .../nacos/common/http/param/Header.java | 16 +- .../nacos/common/http/param/MediaType.java | 80 ++- .../nacos/common/http/param/Query.java | 8 +- .../nacos/common/model/RequestHttpEntity.java | 3 +- .../common/notify/DefaultSharePublisher.java | 4 + .../nacos/common/utils/ConvertUtils.java | 20 +- .../nacos/common/utils/HttpMethod.java | 12 +- .../alibaba/nacos/common/utils/IoUtils.java | 6 +- .../common/http/param/MediaTypeTest.java | 62 ++ .../nacos/common/notify/NotifyCenterTest.java | 1 + .../ConditionDistributedEmbedStorage.java | 2 +- .../ConditionStandaloneEmbedStorage.java | 2 +- .../server/controller/ConfigController.java | 39 +- .../controller/ConfigOpsController.java | 2 +- .../server/controller/HealthController.java | 4 +- .../server/model/app/ApplicationInfo.java | 6 +- .../server/service/ConfigChangePublisher.java | 2 +- .../server/service/ConfigSubService.java | 2 +- .../ExternalDataSourceServiceImpl.java | 2 +- .../LocalDataSourceServiceImpl.java | 4 +- .../server/service/dump/DumpService.java | 6 +- .../service/dump/EmbeddedDumpService.java | 2 +- .../service/merge/MergeDatumService.java | 8 +- .../service/merge/MergeTaskProcessor.java | 6 +- .../service/notify/AsyncNotifyService.java | 14 +- .../service/notify/NotifyTaskProcessor.java | 12 +- .../embedded/DerbySnapshotOperation.java | 4 +- .../DistributedDatabaseOperateImpl.java | 2 +- .../StandaloneDatabaseOperateImpl.java | 2 +- .../ExternalStoragePaginationHelperImpl.java | 2 +- .../ExternalStoragePersistServiceImpl.java | 2 +- .../sql/EmbeddedStorageContextUtils.java | 2 +- .../service/trace/ConfigTraceService.java | 12 +- .../nacos/config/server/utils/DiskUtil.java | 2 +- .../config/server/utils/NamespaceUtil.java | 47 ++ .../config/server/utils/PropertyUtil.java | 2 +- .../config/server/model/ConfigInfoTest.java | 2 +- .../service/ConfigChangePublisherTest.java | 2 +- .../server/utils/NamespaceUtilTest.java | 47 ++ .../nacos/consistency/SerializeFactory.java | 3 + .../alibaba/nacos/consistency/Serializer.java | 13 +- .../serialize/HessianSerializer.java | 32 +- .../serialize/JacksonSerializer.java | 62 ++ .../com.alibaba.nacos.consistency.Serializer | 3 +- .../controller/ServerStateController.java | 2 +- .../src/main/resources/application.properties | 1 + .../alibaba/nacos/core/auth/AuthFilter.java | 2 +- .../auth/condition/PathRequestCondition.java | 2 +- .../alibaba/nacos/core/cluster/Member.java | 2 +- .../core/cluster/MemberMetaDataConstants.java | 6 +- .../nacos/core/cluster/MemberUtils.java | 29 +- .../core/cluster/ServerMemberManager.java | 34 +- .../lookup/AddressServerMemberLookup.java | 3 +- .../lookup/FileConfigMemberLookup.java | 8 +- .../core/cluster/lookup/LookupFactory.java | 2 +- .../lookup/StandaloneMemberLookup.java | 6 +- .../core/code/ControllerMethodsCache.java | 3 +- .../LoggingSpringApplicationRunListener.java | 2 +- .../StandaloneProfileApplicationListener.java | 6 +- .../StartingSpringApplicationRunListener.java | 10 +- .../controller/NacosClusterController.java | 2 +- .../core/distributed/ProtocolManager.java | 2 +- .../distro/task/load/DistroLoadDataTask.java | 2 +- .../distributed/id/SnowFlowerIdGenerator.java | 2 +- .../core/distributed/raft/JRaftServer.java | 12 +- .../distributed/raft/utils/JRaftUtils.java | 4 +- .../nacos/core/storage/kv/FileKvStorage.java | 34 +- .../nacos/core/storage/kv/KvStorage.java | 8 + .../core/storage/kv/MemoryKvStorage.java | 16 +- .../nacos/core/utils/PropertyUtil.java | 81 --- .../nacos/core/utils/TimerContext.java | 2 + .../alibaba/nacos/core/utils/WebUtils.java | 1 + .../main/resources/META-INF/logback/nacos.xml | 2 +- .../main/resources/META-INF/spring.factories | 9 - .../nacos/core/cluster/MemberUtilsTest.java | 106 ++++ .../core/cluster/ServerMemberManagerTest.java | 111 ++++ ...ndaloneProfileApplicationListenerTest.java | 2 +- .../nacos/core/storage/RocksStorageTest.java | 76 --- .../nacos/core/utils/SystemUtilsTest.java | 5 +- distribution/conf/nacos-logback.xml | 2 +- .../naming/cluster/ServerListManager.java | 2 +- .../DelegateConsistencyServiceImpl.java | 43 +- .../nacos/naming/consistency/KeyBuilder.java | 9 +- .../distro/DistroConsistencyServiceImpl.java | 21 - .../ephemeral/distro/DistroHttpData.java | 4 +- .../ephemeral/distro/DistroHttpRegistry.java | 77 +++ .../component/DistroDataStorageImpl.java | 2 +- ...sistentConsistencyServiceDelegateImpl.java | 91 +++ .../persistent/PersistentNotifier.java | 31 +- .../persistent/impl/NamingKvStorage.java | 229 +++++++ .../impl/NamingSnapshotOperation.java | 2 +- .../impl/PersistentServiceProcessor.java | 161 ++--- .../raft/RaftConsistencyServiceImpl.java | 14 +- .../consistency/persistent/raft/RaftCore.java | 219 ++++--- .../persistent/raft/RaftPeerSet.java | 40 +- .../persistent/raft/RaftProxy.java | 22 +- .../persistent/raft/RaftStore.java | 34 +- .../naming/controllers/ApiController.java | 6 +- .../naming/controllers/DistroController.java | 5 +- .../naming/controllers/HealthController.java | 2 +- .../controllers/InstanceController.java | 48 +- .../controllers/OperatorController.java | 2 +- .../naming/controllers/RaftController.java | 8 +- .../nacos/naming/core/DistroMapper.java | 2 +- .../alibaba/nacos/naming/core/Service.java | 10 +- .../nacos/naming/core/ServiceManager.java | 29 +- .../nacos/naming/core/SubscribeManager.java | 10 +- .../healthcheck/ClientBeatCheckTask.java | 33 +- .../healthcheck/ClientBeatProcessor.java | 2 +- .../naming/healthcheck/HealthCheckCommon.java | 8 +- .../healthcheck/HealthCheckReactor.java | 1 + .../naming/healthcheck/HealthCheckTask.java | 2 +- .../alibaba/nacos/naming/misc/HttpClient.java | 600 ++++-------------- .../nacos/naming/misc/HttpClientManager.java | 149 +++++ .../nacos/naming/misc/NamingProxy.java | 100 +-- .../alibaba/nacos/naming/misc/NetUtils.java | 6 +- .../naming/misc/ServerStatusSynchronizer.java | 25 +- .../misc/ServiceStatusSynchronizer.java | 27 +- .../nacos/naming/misc/UtilsAndCommons.java | 4 +- .../monitor/PerformanceLoggerThread.java | 26 +- .../nacos/naming/selector/LabelSelector.java | 2 +- .../nacos/naming/web/DistroFilter.java | 6 +- .../nacos/naming/web/TrafficReviseFilter.java | 2 +- .../com/alibaba/nacos/naming/BaseTest.java | 2 +- .../com/alibaba/nacos/naming/JraftTest.java | 32 + .../DistroConsistencyServiceImplTest.java | 11 - .../ClusterVersionJudgementTest.java | 2 +- .../impl/NamingSnapshotOperationTest.java | 13 +- .../controllers/ClusterControllerTest.java | 35 +- .../nacos/naming/core/ClusterTest.java | 2 +- pom.xml | 8 +- style/nacos-code-style-for-idea.xml | 1 + sys/pom.xml | 90 +++ .../com/alibaba/nacos/sys/env}/Constants.java | 2 +- .../NacosAutoRefreshPropertySourceLoader.java | 94 +++ ...ropertySourceEnvironmentPostProcessor.java | 9 +- .../env/OriginTrackedPropertiesLoader.java | 259 ++++++++ .../nacos/sys}/file/FileChangeEvent.java | 2 +- .../alibaba/nacos/sys}/file/FileWatcher.java | 2 +- .../nacos/sys}/file/WatchFileCenter.java | 4 +- .../nacos/sys}/utils/ApplicationUtils.java | 40 +- .../alibaba/nacos/sys}/utils/DiskUtils.java | 168 ++--- .../alibaba/nacos/sys}/utils/InetUtils.java | 168 ++--- .../META-INF/nacos-default.properties | 4 +- .../main/resources/META-INF/spring.factories | 9 + ...osAutoRefreshPropertySourceLoaderTest.java | 92 +++ ...rtySourceEnvironmentPostProcessorTest.java | 6 +- sys/src/test/resources/application.properties | 17 + .../test/common/WatchFileCenter_ITCase.java | 282 ++++---- .../nacos/test/config/ConfigCleanUtils.java | 2 +- .../config/ConfigDerbyImport_CITCase.java | 6 +- .../test/config/ConfigDerbyRaft_DITCase.java | 8 +- .../nacos/test/core/BaseClusterTest.java | 9 +- .../nacos/test/core/InetUtils_ITCase.java | 13 +- .../core/SnowFlowerIdGenerator_ITCase.java | 2 +- .../test/core/auth/NamingAuth_ITCase.java | 2 +- .../core/cluster/MemberLookup_ITCase.java | 8 +- .../cluster/ServerMemberManager_ITCase.java | 4 +- .../naming/AutoDeregisterInstance_ITCase.java | 2 +- .../nacos/test/naming/ClientBeat_ITCase.java | 2 +- .../naming/DeregisterInstance_ITCase.java | 2 +- .../nacos/test/naming/MultiTenant_ITCase.java | 2 +- .../naming/NamingMaintainService_ITCase.java | 2 +- .../test/naming/RegisterInstance_ITCase.java | 2 +- .../test/naming/SelectInstances_ITCase.java | 2 +- .../SelectOneHealthyInstance_ITCase.java | 2 +- .../test/naming/ServiceListTest_ITCase.java | 2 +- .../test/naming/SubscribeCluster_ITCase.java | 2 +- .../nacos/test/naming/Unsubscribe_ITCase.java | 2 +- 195 files changed, 3272 insertions(+), 2042 deletions(-) delete mode 100644 auth/src/main/java/com/alibaba/nacos/auth/common/env/ReloadableConfigs.java delete mode 100644 client/src/main/resources/application.properties delete mode 100644 client/src/test/java/com/alibaba/nacos/client/config/listener/impl/ClientWorkerTest.java create mode 100644 common/src/test/java/com/alibaba/nacos/common/http/param/MediaTypeTest.java create mode 100644 config/src/main/java/com/alibaba/nacos/config/server/utils/NamespaceUtil.java create mode 100644 config/src/test/java/com/alibaba/nacos/config/server/utils/NamespaceUtilTest.java create mode 100644 consistency/src/main/java/com/alibaba/nacos/consistency/serialize/JacksonSerializer.java rename api/src/main/resources/application.properties => consistency/src/main/resources/META-INF/services/com.alibaba.nacos.consistency.Serializer (91%) delete mode 100644 core/src/main/java/com/alibaba/nacos/core/utils/PropertyUtil.java create mode 100644 core/src/test/java/com/alibaba/nacos/core/cluster/MemberUtilsTest.java create mode 100644 core/src/test/java/com/alibaba/nacos/core/cluster/ServerMemberManagerTest.java delete mode 100644 core/src/test/java/com/alibaba/nacos/core/storage/RocksStorageTest.java create mode 100644 naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroHttpRegistry.java create mode 100644 naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/PersistentConsistencyServiceDelegateImpl.java create mode 100644 naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingKvStorage.java create mode 100644 naming/src/main/java/com/alibaba/nacos/naming/misc/HttpClientManager.java create mode 100644 naming/src/test/java/com/alibaba/nacos/naming/JraftTest.java create mode 100644 sys/pom.xml rename {core/src/main/java/com/alibaba/nacos/core/utils => sys/src/main/java/com/alibaba/nacos/sys/env}/Constants.java (98%) create mode 100644 sys/src/main/java/com/alibaba/nacos/sys/env/NacosAutoRefreshPropertySourceLoader.java rename {core/src/main/java/com/alibaba/nacos/core => sys/src/main/java/com/alibaba/nacos/sys}/env/NacosDefaultPropertySourceEnvironmentPostProcessor.java (94%) create mode 100644 sys/src/main/java/com/alibaba/nacos/sys/env/OriginTrackedPropertiesLoader.java rename {core/src/main/java/com/alibaba/nacos/core => sys/src/main/java/com/alibaba/nacos/sys}/file/FileChangeEvent.java (98%) rename {core/src/main/java/com/alibaba/nacos/core => sys/src/main/java/com/alibaba/nacos/sys}/file/FileWatcher.java (97%) rename {core/src/main/java/com/alibaba/nacos/core => sys/src/main/java/com/alibaba/nacos/sys}/file/WatchFileCenter.java (99%) rename {core/src/main/java/com/alibaba/nacos/core => sys/src/main/java/com/alibaba/nacos/sys}/utils/ApplicationUtils.java (93%) rename {core/src/main/java/com/alibaba/nacos/core => sys/src/main/java/com/alibaba/nacos/sys}/utils/DiskUtils.java (75%) rename {core/src/main/java/com/alibaba/nacos/core => sys/src/main/java/com/alibaba/nacos/sys}/utils/InetUtils.java (57%) rename {core => sys}/src/main/resources/META-INF/nacos-default.properties (85%) create mode 100644 sys/src/main/resources/META-INF/spring.factories create mode 100644 sys/src/test/java/com/alibaba/nacos/sys/env/NacosAutoRefreshPropertySourceLoaderTest.java rename {core/src/test/java/com/alibaba/nacos/core => sys/src/test/java/com/alibaba/nacos/sys}/env/NacosDefaultPropertySourceEnvironmentPostProcessorTest.java (93%) create mode 100644 sys/src/test/resources/application.properties diff --git a/api/src/main/java/com/alibaba/nacos/api/naming/utils/NamingUtils.java b/api/src/main/java/com/alibaba/nacos/api/naming/utils/NamingUtils.java index 911219ae3..9faa8b40b 100644 --- a/api/src/main/java/com/alibaba/nacos/api/naming/utils/NamingUtils.java +++ b/api/src/main/java/com/alibaba/nacos/api/naming/utils/NamingUtils.java @@ -28,6 +28,9 @@ import com.alibaba.nacos.api.utils.StringUtils; public class NamingUtils { public static String getGroupedName(final String serviceName, final String groupName) { + if (StringUtils.isBlank(serviceName)) { + throw new IllegalArgumentException("Param 'serviceName' is illegal, serviceName is blank"); + } final String resultGroupedName = groupName + Constants.SERVICE_INFO_SPLITER + serviceName; return resultGroupedName.intern(); } diff --git a/auth/pom.xml b/auth/pom.xml index a048b81cf..afd43988b 100644 --- a/auth/pom.xml +++ b/auth/pom.xml @@ -40,6 +40,10 @@ ${project.groupId} nacos-common + + ${project.groupId} + nacos-sys + org.springframework.boot diff --git a/auth/src/main/java/com/alibaba/nacos/auth/common/AuthConfigs.java b/auth/src/main/java/com/alibaba/nacos/auth/common/AuthConfigs.java index 19638654b..721359b36 100644 --- a/auth/src/main/java/com/alibaba/nacos/auth/common/AuthConfigs.java +++ b/auth/src/main/java/com/alibaba/nacos/auth/common/AuthConfigs.java @@ -16,12 +16,11 @@ package com.alibaba.nacos.auth.common; -import com.alibaba.nacos.auth.common.env.ReloadableConfigs; import com.alibaba.nacos.common.JustForTest; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import io.jsonwebtoken.io.Decoders; import org.apache.commons.lang3.BooleanUtils; import org.apache.commons.lang3.StringUtils; -import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Configuration; @@ -40,9 +39,6 @@ public class AuthConfigs { @JustForTest private static Boolean cachingEnabled = null; - @Autowired - private ReloadableConfigs reloadableConfigs; - /** * secret key. */ @@ -93,7 +89,7 @@ public class AuthConfigs { return BooleanUtils.toBoolean(enabled); } return BooleanUtils - .toBoolean(reloadableConfigs.getProperties().getProperty("nacos.core.auth.enabled", "false")); + .toBoolean(ApplicationUtils.getProperty("nacos.core.auth.enabled", "false")); } /** @@ -106,7 +102,7 @@ public class AuthConfigs { return cachingEnabled; } return BooleanUtils - .toBoolean(reloadableConfigs.getProperties().getProperty("nacos.core.auth.caching.enabled", "true")); + .toBoolean(ApplicationUtils.getProperty("nacos.core.auth.caching.enabled", "true")); } @JustForTest diff --git a/auth/src/main/java/com/alibaba/nacos/auth/common/env/ReloadableConfigs.java b/auth/src/main/java/com/alibaba/nacos/auth/common/env/ReloadableConfigs.java deleted file mode 100644 index 6984ad52e..000000000 --- a/auth/src/main/java/com/alibaba/nacos/auth/common/env/ReloadableConfigs.java +++ /dev/null @@ -1,75 +0,0 @@ -/* - * Copyright 1999-2018 Alibaba Group Holding Ltd. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.nacos.auth.common.env; - -import org.apache.commons.lang3.StringUtils; -import org.springframework.beans.factory.annotation.Value; -import org.springframework.scheduling.annotation.Scheduled; -import org.springframework.stereotype.Component; - -import java.io.File; -import java.io.FileInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.util.Properties; - -/** - * Reload application.properties. - * - * @author nkorange - * @author mai.jh - * @since 1.2.0 - */ -@Component -public class ReloadableConfigs { - - private static final String FILE_PREFIX = "file:"; - - private Properties properties; - - @Value("${spring.config.location:}") - private String path; - - /** - * Periodically load configuration file information. - * - * @throws IOException IOException - */ - @Scheduled(fixedRate = 5000) - public void reload() throws IOException { - final Properties properties = new Properties(); - InputStream inputStream = null; - if (StringUtils.isNotBlank(path) && path.contains(FILE_PREFIX)) { - String[] paths = path.split(","); - path = paths[paths.length - 1].substring(FILE_PREFIX.length()); - } - try { - inputStream = new FileInputStream(new File(path + "application.properties")); - } catch (Exception ignore) { - } - if (inputStream == null) { - inputStream = getClass().getResourceAsStream("/application.properties"); - } - properties.load(inputStream); - inputStream.close(); - this.properties = properties; - } - - public final Properties getProperties() { - return properties; - } -} diff --git a/client/src/main/java/com/alibaba/nacos/client/config/impl/ClientWorker.java b/client/src/main/java/com/alibaba/nacos/client/config/impl/ClientWorker.java index fab00f032..93578dda9 100644 --- a/client/src/main/java/com/alibaba/nacos/client/config/impl/ClientWorker.java +++ b/client/src/main/java/com/alibaba/nacos/client/config/impl/ClientWorker.java @@ -32,7 +32,6 @@ import com.alibaba.nacos.client.utils.ParamUtil; import com.alibaba.nacos.client.utils.TenantUtil; import com.alibaba.nacos.common.http.HttpRestResult; import com.alibaba.nacos.common.lifecycle.Closeable; -import com.alibaba.nacos.common.utils.ConcurrentHashSet; import com.alibaba.nacos.common.utils.ConvertUtils; import com.alibaba.nacos.common.utils.MD5Utils; import com.alibaba.nacos.common.utils.StringUtils; @@ -166,7 +165,6 @@ public class ClientWorker implements Closeable { copy.remove(groupKey); cacheMap.set(copy); } - reMakeCacheDataTaskId(); LOGGER.info("[{}] [unsubscribe] {}", this.agent.getName(), groupKey); MetricsMonitor.getListenConfigCountMonitor().set(cacheMap.get().size()); @@ -179,38 +177,11 @@ public class ClientWorker implements Closeable { copy.remove(groupKey); cacheMap.set(copy); } - reMakeCacheDataTaskId(); LOGGER.info("[{}] [unsubscribe] {}", agent.getName(), groupKey); MetricsMonitor.getListenConfigCountMonitor().set(cacheMap.get().size()); } - /** - * Remake cacheData taskId. - */ - private void reMakeCacheDataTaskId() { - int listenerSize = cacheMap.get().size(); - int remakeTaskId = (int) Math.ceil(listenerSize / ParamUtil.getPerTaskConfigSize()); - if (remakeTaskId < (int) currentLongingTaskCount) { - for (int i = 0; i < remakeTaskId; i++) { - int count = 0; - for (String key : cacheMap.get().keySet()) { - if (count == ParamUtil.getPerTaskConfigSize()) { - break; - } - CacheData cacheData = cacheMap.get().get(key); - cacheData.setTaskId(i); - synchronized (cacheMap) { - Map copy = new HashMap(this.cacheMap.get()); - copy.put(key, cacheData); - cacheMap.set(copy); - } - count++; - } - } - } - } - /** * Add cache data if absent. * @@ -277,8 +248,6 @@ public class ClientWorker implements Closeable { cache.setInitializing(true); } else { cache = new CacheData(configFilterChainManager, agent.getName(), dataId, group, tenant); - int taskId = cacheMap.get().size() / (int) ParamUtil.getPerTaskConfigSize(); - cache.setTaskId(taskId); // fix issue # 1317 if (enableRemoteSyncConfig) { String[] ct = getServerConfig(dataId, group, tenant, 3000L); @@ -425,16 +394,11 @@ public class ClientWorker implements Closeable { int longingTaskCount = (int) Math.ceil(listenerSize / ParamUtil.getPerTaskConfigSize()); if (longingTaskCount > currentLongingTaskCount) { for (int i = (int) currentLongingTaskCount; i < longingTaskCount; i++) { - taskIdSet.add(i); // The task list is no order.So it maybe has issues when changing. executorService.execute(new LongPollingRunnable(i)); } - } else if (longingTaskCount < currentLongingTaskCount) { - for (int i = longingTaskCount; i < (int) currentLongingTaskCount; i++) { - taskIdSet.remove(i); - } + currentLongingTaskCount = longingTaskCount; } - currentLongingTaskCount = longingTaskCount; } /** @@ -692,9 +656,7 @@ public class ClientWorker implements Closeable { } inInitializingCacheList.clear(); - if (taskIdSet.contains(taskId)) { - executorService.execute(this); - } + executorService.execute(this); } catch (Throwable e) { @@ -723,11 +685,6 @@ public class ClientWorker implements Closeable { private final AtomicReference> cacheMap = new AtomicReference>( new HashMap()); - /** - * Store the running taskId. - */ - private final ConcurrentHashSet taskIdSet = new ConcurrentHashSet(); - private final HttpAgent agent; private final ConfigFilterChainManager configFilterChainManager; diff --git a/client/src/main/java/com/alibaba/nacos/client/logging/AbstractNacosLogging.java b/client/src/main/java/com/alibaba/nacos/client/logging/AbstractNacosLogging.java index 4d5a78db6..fa7427748 100644 --- a/client/src/main/java/com/alibaba/nacos/client/logging/AbstractNacosLogging.java +++ b/client/src/main/java/com/alibaba/nacos/client/logging/AbstractNacosLogging.java @@ -19,6 +19,8 @@ package com.alibaba.nacos.client.logging; import com.alibaba.nacos.common.utils.ConvertUtils; import com.alibaba.nacos.common.utils.StringUtils; +import java.io.File; + /** * Abstract nacos logging. * @@ -31,13 +33,13 @@ public abstract class AbstractNacosLogging { private static final String NACOS_LOGGING_DEFAULT_CONFIG_ENABLED_PROPERTY = "nacos.logging.default.config.enabled"; - private static final String NACOS_LOGGING_PATH_PROPERTY = "nacos.logging.path"; + private static final String NACOS_LOGGING_PATH_PROPERTY = "JM.LOG.PATH"; static { String loggingPath = System.getProperty(NACOS_LOGGING_PATH_PROPERTY); if (StringUtils.isBlank(loggingPath)) { String userHome = System.getProperty("user.home"); - System.setProperty(NACOS_LOGGING_PATH_PROPERTY, userHome + "/logs/nacos"); + System.setProperty(NACOS_LOGGING_PATH_PROPERTY, userHome + File.separator + "logs"); } } diff --git a/client/src/main/java/com/alibaba/nacos/client/naming/NacosNamingService.java b/client/src/main/java/com/alibaba/nacos/client/naming/NacosNamingService.java index 19ec45d4d..0ba34aa64 100644 --- a/client/src/main/java/com/alibaba/nacos/client/naming/NacosNamingService.java +++ b/client/src/main/java/com/alibaba/nacos/client/naming/NacosNamingService.java @@ -39,6 +39,7 @@ import com.alibaba.nacos.client.utils.ValidatorUtils; import com.alibaba.nacos.common.utils.ConvertUtils; import com.alibaba.nacos.common.utils.StringUtils; +import java.io.File; import java.util.ArrayList; import java.util.Iterator; import java.util.List; @@ -151,9 +152,13 @@ public class NacosNamingService implements NamingService { } private void initCacheDir() { - cacheDir = System.getProperty("com.alibaba.nacos.naming.cache.dir"); - if (StringUtils.isEmpty(cacheDir)) { - cacheDir = System.getProperty("user.home") + "/nacos/naming/" + namespace; + String jmSnapshotPath = System.getProperty("JM.SNAPSHOT.PATH"); + if (!StringUtils.isBlank(jmSnapshotPath)) { + cacheDir = jmSnapshotPath + File.separator + "nacos" + File.separator + "naming" + + File.separator + namespace; + } else { + cacheDir = System.getProperty("user.home") + File.separator + "nacos" + File.separator + "naming" + + File.separator + namespace; } } diff --git a/client/src/main/java/com/alibaba/nacos/client/naming/cache/ConcurrentDiskUtil.java b/client/src/main/java/com/alibaba/nacos/client/naming/cache/ConcurrentDiskUtil.java index 77eb2bad4..d227a6edd 100644 --- a/client/src/main/java/com/alibaba/nacos/client/naming/cache/ConcurrentDiskUtil.java +++ b/client/src/main/java/com/alibaba/nacos/client/naming/cache/ConcurrentDiskUtil.java @@ -143,11 +143,12 @@ public class ConcurrentDiskUtil { } } while (null == lock); - ByteBuffer sendBuffer = ByteBuffer.wrap(content.getBytes(charsetName)); + byte[] contentBytes = content.getBytes(charsetName); + ByteBuffer sendBuffer = ByteBuffer.wrap(contentBytes); while (sendBuffer.hasRemaining()) { channel.write(sendBuffer); } - channel.truncate(content.length()); + channel.truncate(contentBytes.length); } catch (FileNotFoundException e) { throw new IOException("file not exist"); } finally { diff --git a/client/src/main/java/com/alibaba/nacos/client/naming/core/PushReceiver.java b/client/src/main/java/com/alibaba/nacos/client/naming/core/PushReceiver.java index 625fb12d2..3a0776d46 100644 --- a/client/src/main/java/com/alibaba/nacos/client/naming/core/PushReceiver.java +++ b/client/src/main/java/com/alibaba/nacos/client/naming/core/PushReceiver.java @@ -107,6 +107,9 @@ public class PushReceiver implements Runnable, Closeable { udpSocket.send(new DatagramPacket(ack.getBytes(UTF_8), ack.getBytes(UTF_8).length, packet.getSocketAddress())); } catch (Exception e) { + if (closed) { + return; + } NAMING_LOGGER.error("[NA] error while receiving push data", e); } } diff --git a/client/src/main/java/com/alibaba/nacos/client/utils/ParamUtil.java b/client/src/main/java/com/alibaba/nacos/client/utils/ParamUtil.java index bbab52593..b3c04043e 100644 --- a/client/src/main/java/com/alibaba/nacos/client/utils/ParamUtil.java +++ b/client/src/main/java/com/alibaba/nacos/client/utils/ParamUtil.java @@ -20,9 +20,9 @@ import com.alibaba.nacos.api.PropertyKeyConst; import com.alibaba.nacos.api.SystemPropertyKeyConst; import com.alibaba.nacos.api.common.Constants; import com.alibaba.nacos.common.utils.StringUtils; +import com.alibaba.nacos.common.utils.VersionUtils; import org.slf4j.Logger; -import java.io.InputStream; import java.util.Properties; import java.util.concurrent.Callable; import java.util.regex.Pattern; @@ -80,19 +80,7 @@ public class ParamUtil { } LOGGER.info("[settings] [http-client] connect timeout:{}", connectTimeout); - try { - InputStream in = ValidatorUtils.class.getClassLoader().getResourceAsStream("application.properties"); - Properties props = new Properties(); - props.load(in); - String val = null; - val = props.getProperty("version"); - if (val != null) { - clientVersion = val; - } - LOGGER.info("NACOS_CLIENT_VERSION: {}", clientVersion); - } catch (Exception e) { - LOGGER.error("[500] read application.properties", e); - } + clientVersion = VersionUtils.version; try { perTaskConfigSize = Double.valueOf(System.getProperty("PER_TASK_CONFIG_SIZE", "3000")); diff --git a/client/src/main/resources/application.properties b/client/src/main/resources/application.properties deleted file mode 100644 index 0b3684628..000000000 --- a/client/src/main/resources/application.properties +++ /dev/null @@ -1,16 +0,0 @@ -# -# Copyright 1999-2018 Alibaba Group Holding Ltd. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -# -version=${project.version} diff --git a/client/src/main/resources/nacos-log4j2.xml b/client/src/main/resources/nacos-log4j2.xml index bf325e9ea..4c431c439 100644 --- a/client/src/main/resources/nacos-log4j2.xml +++ b/client/src/main/resources/nacos-log4j2.xml @@ -17,8 +17,8 @@ - + %d{yyyy-MM-dd HH:mm:ss.SSS} %p [%-5t:%c{2}] %m%n @@ -31,8 +31,8 @@ - + %d{yyyy-MM-dd HH:mm:ss.SSS} %p [%-5t:%c{2}] %m%n diff --git a/client/src/main/resources/nacos-logback.xml b/client/src/main/resources/nacos-logback.xml index eb5589632..237700a1e 100644 --- a/client/src/main/resources/nacos-logback.xml +++ b/client/src/main/resources/nacos-logback.xml @@ -19,10 +19,10 @@ nacos - ${nacos.logging.path}/config.log + ${JM.LOG.PATH}/nacos/config.log - ${nacos.logging.path}/config.log.%i + ${JM.LOG.PATH}/nacos/config.log.%i ${JM.LOG.RETAIN.COUNT:-7} @@ -36,10 +36,10 @@ - ${nacos.logging.path}/naming.log + ${JM.LOG.PATH}/nacos/naming.log - ${nacos.logging.path}/naming.log.%i + ${JM.LOG.PATH}/nacos/naming.log.%i ${JM.LOG.RETAIN.COUNT:-7} diff --git a/client/src/test/java/com/alibaba/nacos/client/config/listener/impl/ClientWorkerTest.java b/client/src/test/java/com/alibaba/nacos/client/config/listener/impl/ClientWorkerTest.java deleted file mode 100644 index 792212418..000000000 --- a/client/src/test/java/com/alibaba/nacos/client/config/listener/impl/ClientWorkerTest.java +++ /dev/null @@ -1,132 +0,0 @@ -/* - * Copyright 1999-2018 Alibaba Group Holding Ltd. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.nacos.client.config.listener.impl; - -import com.alibaba.nacos.api.config.listener.Listener; -import com.alibaba.nacos.api.exception.NacosException; -import com.alibaba.nacos.client.config.filter.impl.ConfigFilterChainManager; -import com.alibaba.nacos.client.config.http.MetricsHttpAgent; -import com.alibaba.nacos.client.config.impl.ClientWorker; -import com.alibaba.nacos.client.utils.ParamUtil; -import org.junit.Assert; -import org.junit.Before; -import org.junit.Test; -import org.mockito.Mock; -import org.mockito.MockitoAnnotations; - -import java.lang.reflect.Field; -import java.lang.reflect.Modifier; -import java.util.Arrays; -import java.util.List; -import java.util.Properties; -import java.util.concurrent.Executor; -import java.util.concurrent.ScheduledExecutorService; - -import static org.mockito.Mockito.mock; - -public class ClientWorkerTest { - - @Mock - ScheduledExecutorService scheduledExecutorService; - - private ClientWorker clientWorker; - - private List listeners; - - private final String dataId = "data"; - - private final String group = "group"; - - private final String currentLongingTaskCount = "currentLongingTaskCount"; - - @Before - public void init() { - MockitoAnnotations.initMocks(this); - clientWorker = new ClientWorker(mock(MetricsHttpAgent.class), mock(ConfigFilterChainManager.class), - mock(Properties.class)); - try { - Field executorServiceField = clientWorker.getClass().getDeclaredField("executorService"); - executorServiceField.setAccessible(true); - Field modifiersField = Field.class.getDeclaredField("modifiers"); - modifiersField.setAccessible(true); - modifiersField.setInt(executorServiceField, executorServiceField.getModifiers() & ~Modifier.FINAL); - executorServiceField.set(clientWorker, scheduledExecutorService); - Listener listener = new Listener() { - @Override - public Executor getExecutor() { - return null; - } - - @Override - public void receiveConfigInfo(String configInfo) { - - } - }; - listeners = Arrays.asList(listener); - } catch (NoSuchFieldException e) { - e.printStackTrace(); - } catch (IllegalAccessException e) { - e.printStackTrace(); - } - } - - @Test - public void testAddLongPollNumberThreads() { - try { - for (int i = 0; i < ParamUtil.getPerTaskConfigSize(); i++) { - clientWorker.addTenantListeners(dataId + i, group, listeners); - } - Field currentLongingTaskCountField = clientWorker.getClass().getDeclaredField(currentLongingTaskCount); - currentLongingTaskCountField.setAccessible(true); - Assert.assertEquals(currentLongingTaskCount, (int) currentLongingTaskCountField.getDouble(clientWorker), 1); - for (int i = (int) ParamUtil.getPerTaskConfigSize(); i < ParamUtil.getPerTaskConfigSize() * 2; i++) { - clientWorker.addTenantListeners(dataId + i, group, listeners); - } - Assert.assertEquals(currentLongingTaskCount, (int) currentLongingTaskCountField.getDouble(clientWorker), 2); - } catch (NacosException e) { - e.printStackTrace(); - } catch (IllegalAccessException e) { - e.printStackTrace(); - } catch (NoSuchFieldException e) { - e.printStackTrace(); - } - } - - @Test - public void testReduceLongPollNumberThreads() { - try { - for (int i = 0; i < ParamUtil.getPerTaskConfigSize() * 3; i++) { - clientWorker.addTenantListeners(dataId + i, group, listeners); - } - Field currentLongingTaskCountField = clientWorker.getClass().getDeclaredField(currentLongingTaskCount); - currentLongingTaskCountField.setAccessible(true); - Assert.assertEquals(currentLongingTaskCount, (int) currentLongingTaskCountField.getDouble(clientWorker), 3); - - for (int i = (int) ParamUtil.getPerTaskConfigSize(); i < ParamUtil.getPerTaskConfigSize() * 2; i++) { - clientWorker.removeTenantListener(dataId + i, group, listeners.get(0)); - } - Assert.assertEquals(currentLongingTaskCount, (int) currentLongingTaskCountField.getDouble(clientWorker), 2); - } catch (NacosException e) { - e.printStackTrace(); - } catch (IllegalAccessException e) { - e.printStackTrace(); - } catch (NoSuchFieldException e) { - e.printStackTrace(); - } - } - -} diff --git a/client/src/test/java/com/alibaba/nacos/client/naming/cache/DiskCacheTest.java b/client/src/test/java/com/alibaba/nacos/client/naming/cache/DiskCacheTest.java index 6079e4da1..6ec35843b 100644 --- a/client/src/test/java/com/alibaba/nacos/client/naming/cache/DiskCacheTest.java +++ b/client/src/test/java/com/alibaba/nacos/client/naming/cache/DiskCacheTest.java @@ -47,6 +47,7 @@ public class DiskCacheTest { instance.setIp("1.1.1.1"); instance.setPort(1234); instance.setServiceName("testName"); + instance.addMetadata("chinese", "中文"); serviceInfo.setHosts(Collections.singletonList(instance)); } @@ -87,9 +88,10 @@ public class DiskCacheTest { } private void assertInstance(Instance actual, Instance expected) { - assertEquals(actual.getServiceName(), actual.getServiceName()); - assertEquals(actual.getClusterName(), actual.getClusterName()); - assertEquals(actual.getIp(), actual.getIp()); - assertEquals(actual.getPort(), actual.getPort()); + assertEquals(actual.getServiceName(), expected.getServiceName()); + assertEquals(actual.getClusterName(), expected.getClusterName()); + assertEquals(actual.getIp(), expected.getIp()); + assertEquals(actual.getPort(), expected.getPort()); + assertEquals(actual.getMetadata(), expected.getMetadata()); } } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/AbstractApacheHttpClientFactory.java b/common/src/main/java/com/alibaba/nacos/common/http/AbstractApacheHttpClientFactory.java index 1d3c32e39..5d70ef14c 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/AbstractApacheHttpClientFactory.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/AbstractApacheHttpClientFactory.java @@ -30,9 +30,15 @@ public abstract class AbstractApacheHttpClientFactory extends AbstractHttpClient @Override public final NacosRestTemplate createNacosRestTemplate() { + final HttpClientConfig originalRequestConfig = buildHttpClientConfig(); final RequestConfig requestConfig = getRequestConfig(); - return new NacosRestTemplate(assignLogger(), - new DefaultHttpClientRequest(HttpClients.custom().setDefaultRequestConfig(requestConfig).build())); + return new NacosRestTemplate(assignLogger(), new DefaultHttpClientRequest( + HttpClients.custom().setDefaultRequestConfig(requestConfig) + .setUserAgent(originalRequestConfig.getUserAgent()) + .setMaxConnTotal(originalRequestConfig.getMaxConnTotal()) + .setMaxConnPerRoute(originalRequestConfig.getMaxConnPerRoute()) + .setConnectionTimeToLive(originalRequestConfig.getConnTimeToLive(), + originalRequestConfig.getConnTimeToLiveTimeUnit()).build())); } } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/AbstractHttpClientFactory.java b/common/src/main/java/com/alibaba/nacos/common/http/AbstractHttpClientFactory.java index dd85b8744..54b0fd715 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/AbstractHttpClientFactory.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/AbstractHttpClientFactory.java @@ -67,15 +67,20 @@ public abstract class AbstractHttpClientFactory implements HttpClientFactory { @Override public NacosAsyncRestTemplate createNacosAsyncRestTemplate() { - RequestConfig requestConfig = getRequestConfig(); + final HttpClientConfig originalRequestConfig = buildHttpClientConfig(); + final RequestConfig requestConfig = getRequestConfig(); return new NacosAsyncRestTemplate(assignLogger(), new DefaultAsyncHttpClientRequest( - HttpAsyncClients.custom().setDefaultRequestConfig(requestConfig).build())); + HttpAsyncClients.custom().setDefaultRequestConfig(requestConfig) + .setMaxConnTotal(originalRequestConfig.getMaxConnTotal()) + .setMaxConnPerRoute(originalRequestConfig.getMaxConnPerRoute()) + .setUserAgent(originalRequestConfig.getUserAgent()).build())); } protected RequestConfig getRequestConfig() { HttpClientConfig httpClientConfig = buildHttpClientConfig(); return RequestConfig.custom().setConnectTimeout(httpClientConfig.getConTimeOutMillis()) .setSocketTimeout(httpClientConfig.getReadTimeOutMillis()) + .setConnectionRequestTimeout(httpClientConfig.getConnectionRequestTimeout()) .setMaxRedirects(httpClientConfig.getMaxRedirects()).build(); } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/BaseHttpClient.java b/common/src/main/java/com/alibaba/nacos/common/http/BaseHttpClient.java index 7881af077..18bd1782d 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/BaseHttpClient.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/BaseHttpClient.java @@ -16,7 +16,6 @@ package com.alibaba.nacos.common.http; -import com.alibaba.nacos.common.constant.HttpHeaderConsts; import com.alibaba.nacos.common.http.handler.ResponseHandler; import com.alibaba.nacos.common.http.param.Header; import com.alibaba.nacos.common.http.param.Query; @@ -103,7 +102,7 @@ public abstract class BaseHttpClient { final BaseHttpMethod httpMethod = BaseHttpMethod.sourceOf(method); final HttpRequestBase httpRequestBase = httpMethod.init(url); HttpUtils.initRequestHeader(httpRequestBase, header); - HttpUtils.initRequestEntity(httpRequestBase, body, header.getValue(HttpHeaderConsts.CONTENT_TYPE)); + HttpUtils.initRequestEntity(httpRequestBase, body, header); return httpRequestBase; } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/BaseHttpMethod.java b/common/src/main/java/com/alibaba/nacos/common/http/BaseHttpMethod.java index 4ccb2407b..b53cfa36d 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/BaseHttpMethod.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/BaseHttpMethod.java @@ -84,6 +84,16 @@ public enum BaseHttpMethod { } }, + /** + * delete Large request. + */ + DELETE_LARGE(HttpMethod.DELETE_LARGE) { + @Override + protected HttpRequestBase createRequest(String url) { + return new HttpDeleteWithEntity(url); + } + }, + /** * head request. */ @@ -155,6 +165,10 @@ public enum BaseHttpMethod { /** * get Large implemented. + *

+ * Mainly used for GET request parameters are relatively large, can not be placed on the URL, so it needs to be + * placed in the body. + *

*/ public static class HttpGetWithEntity extends HttpEntityEnclosingRequestBase { @@ -171,4 +185,26 @@ public enum BaseHttpMethod { } } + /** + * delete Large implemented. + *

+ * Mainly used for DELETE request parameters are relatively large, can not be placed on the URL, so it needs to be + * placed in the body. + *

+ */ + public static class HttpDeleteWithEntity extends HttpEntityEnclosingRequestBase { + + public static final String METHOD_NAME = "DELETE"; + + public HttpDeleteWithEntity(String url) { + super(); + setURI(URI.create(url)); + } + + @Override + public String getMethod() { + return METHOD_NAME; + } + } + } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/HttpClientConfig.java b/common/src/main/java/com/alibaba/nacos/common/http/HttpClientConfig.java index cc5b9993b..ae50ab485 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/HttpClientConfig.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/HttpClientConfig.java @@ -16,6 +16,8 @@ package com.alibaba.nacos.common.http; +import java.util.concurrent.TimeUnit; + /** * http client config build. * @@ -23,16 +25,62 @@ package com.alibaba.nacos.common.http; */ public class HttpClientConfig { + /** + * connect time out. + */ private final int conTimeOutMillis; + /** + * read time out. + */ private final int readTimeOutMillis; + /** + * connTimeToLive. + */ + private final long connTimeToLive; + + /** + * connTimeToLiveTimeUnit. + */ + private final TimeUnit connTimeToLiveTimeUnit; + + /** + * connectionRequestTimeout. + */ + private final int connectionRequestTimeout; + + /** + * max redirect. + */ private final int maxRedirects; - public HttpClientConfig(int conTimeOutMillis, int readTimeOutMillis, int maxRedirects) { + /** + * max connect total. + */ + private final int maxConnTotal; + + /** + * Assigns maximum connection per route value. + */ + private final int maxConnPerRoute; + + /** + * user agent. + */ + private final String userAgent; + + public HttpClientConfig(int conTimeOutMillis, int readTimeOutMillis, long connTimeToLive, TimeUnit timeUnit, + int connectionRequestTimeout, int maxRedirects, int maxConnTotal, int maxConnPerRoute, String userAgent) { this.conTimeOutMillis = conTimeOutMillis; this.readTimeOutMillis = readTimeOutMillis; + this.connTimeToLive = connTimeToLive; + this.connTimeToLiveTimeUnit = timeUnit; + this.connectionRequestTimeout = connectionRequestTimeout; this.maxRedirects = maxRedirects; + this.maxConnTotal = maxConnTotal; + this.maxConnPerRoute = maxConnPerRoute; + this.userAgent = userAgent; } public int getConTimeOutMillis() { @@ -43,10 +91,34 @@ public class HttpClientConfig { return readTimeOutMillis; } + public long getConnTimeToLive() { + return connTimeToLive; + } + + public TimeUnit getConnTimeToLiveTimeUnit() { + return connTimeToLiveTimeUnit; + } + + public int getConnectionRequestTimeout() { + return connectionRequestTimeout; + } + public int getMaxRedirects() { return maxRedirects; } + public int getMaxConnTotal() { + return maxConnTotal; + } + + public int getMaxConnPerRoute() { + return maxConnPerRoute; + } + + public String getUserAgent() { + return userAgent; + } + public static HttpClientConfigBuilder builder() { return new HttpClientConfigBuilder(); } @@ -57,8 +129,20 @@ public class HttpClientConfig { private int readTimeOutMillis = -1; + private long connTimeToLive = -1; + + private TimeUnit connTimeToLiveTimeUnit = TimeUnit.MILLISECONDS; + + private int connectionRequestTimeout = -1; + private int maxRedirects = 50; + private int maxConnTotal = 0; + + private int maxConnPerRoute = 0; + + private String userAgent; + public HttpClientConfigBuilder setConTimeOutMillis(int conTimeOutMillis) { this.conTimeOutMillis = conTimeOutMillis; return this; @@ -69,13 +153,40 @@ public class HttpClientConfig { return this; } + public HttpClientConfigBuilder setConnectionTimeToLive(long connTimeToLive, TimeUnit connTimeToLiveTimeUnit) { + this.connTimeToLive = connTimeToLive; + this.connTimeToLiveTimeUnit = connTimeToLiveTimeUnit; + return this; + } + + public HttpClientConfigBuilder setConnectionRequestTimeout(int connectionRequestTimeout) { + this.connectionRequestTimeout = connectionRequestTimeout; + return this; + } + public HttpClientConfigBuilder setMaxRedirects(int maxRedirects) { this.maxRedirects = maxRedirects; return this; } + public HttpClientConfigBuilder setMaxConnTotal(int maxConnTotal) { + this.maxConnTotal = maxConnTotal; + return this; + } + + public HttpClientConfigBuilder setMaxConnPerRoute(int maxConnPerRoute) { + this.maxConnPerRoute = maxConnPerRoute; + return this; + } + + public HttpClientConfigBuilder setUserAgent(String userAgent) { + this.userAgent = userAgent; + return this; + } + public HttpClientConfig build() { - return new HttpClientConfig(conTimeOutMillis, readTimeOutMillis, maxRedirects); + return new HttpClientConfig(conTimeOutMillis, readTimeOutMillis, connTimeToLive, connTimeToLiveTimeUnit, + connectionRequestTimeout, maxRedirects, maxConnTotal, maxConnPerRoute, userAgent); } } } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/HttpUtils.java b/common/src/main/java/com/alibaba/nacos/common/http/HttpUtils.java index 97f6f5360..54f99affc 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/HttpUtils.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/HttpUtils.java @@ -16,15 +16,18 @@ package com.alibaba.nacos.common.http; -import com.alibaba.nacos.common.http.handler.RequestHandler; +import com.alibaba.nacos.common.constant.HttpHeaderConsts; import com.alibaba.nacos.common.http.param.Header; +import com.alibaba.nacos.common.http.param.MediaType; import com.alibaba.nacos.common.http.param.Query; +import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.common.utils.StringUtils; import org.apache.http.HttpEntity; import org.apache.http.HttpEntityEnclosingRequest; import org.apache.http.NameValuePair; import org.apache.http.client.entity.UrlEncodedFormEntity; import org.apache.http.client.methods.HttpRequestBase; +import org.apache.http.entity.ByteArrayEntity; import org.apache.http.entity.ContentType; import org.apache.http.entity.StringEntity; import org.apache.http.message.BasicNameValuePair; @@ -70,17 +73,23 @@ public final class HttpUtils { * * @param requestBase requestBase {@link HttpRequestBase} * @param body body - * @param mediaType mediaType {@link ContentType} + * @param header request header * @throws Exception exception */ - public static void initRequestEntity(HttpRequestBase requestBase, Object body, String mediaType) throws Exception { + public static void initRequestEntity(HttpRequestBase requestBase, Object body, Header header) throws Exception { if (body == null) { return; } if (requestBase instanceof HttpEntityEnclosingRequest) { HttpEntityEnclosingRequest request = (HttpEntityEnclosingRequest) requestBase; - ContentType contentType = ContentType.create(mediaType); - StringEntity entity = new StringEntity(RequestHandler.parse(body), contentType); + MediaType mediaType = MediaType.valueOf(header.getValue(HttpHeaderConsts.CONTENT_TYPE)); + ContentType contentType = ContentType.create(mediaType.getType(), mediaType.getCharset()); + HttpEntity entity; + if (body instanceof byte[]) { + entity = new ByteArrayEntity((byte[]) body, contentType); + } else { + entity = new StringEntity(body instanceof String ? (String) body : JacksonUtils.toJson(body), contentType); + } request.setEntity(entity); } } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/client/NacosAsyncRestTemplate.java b/common/src/main/java/com/alibaba/nacos/common/http/client/NacosAsyncRestTemplate.java index 082400113..ad9ff06cb 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/client/NacosAsyncRestTemplate.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/client/NacosAsyncRestTemplate.java @@ -105,6 +105,27 @@ public class NacosAsyncRestTemplate extends AbstractNacosRestTemplate { execute(url, HttpMethod.DELETE, new RequestHttpEntity(header, query), responseType, callback); } + /** + * async http delete large request, when the parameter exceeds the URL limit, you can use this method to put the + * parameter into the body pass. + * + *

{@code responseType} can be an RestResult or RestResult data {@code T} type + * + *

{@code callback} Result callback execution, + * if you need response headers, you can convert the received RestResult to HttpRestResult. + * + * @param url url + * @param header http header param + * @param body body + * @param responseType return type + * @param callback callback {@link Callback#onReceive(com.alibaba.nacos.common.model.RestResult)} + */ + public void delete(String url, Header header, String body, Type responseType, Callback callback) { + execute(url, HttpMethod.DELETE_LARGE, + new RequestHttpEntity(header.setContentType(MediaType.APPLICATION_JSON), Query.EMPTY, body), + responseType, callback); + } + /** * async http put Create a new resource by PUTting the given body to http request. * diff --git a/common/src/main/java/com/alibaba/nacos/common/http/client/NacosRestTemplate.java b/common/src/main/java/com/alibaba/nacos/common/http/client/NacosRestTemplate.java index f7b3693a5..1d60c2e4c 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/client/NacosRestTemplate.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/client/NacosRestTemplate.java @@ -427,6 +427,26 @@ public class NacosRestTemplate extends AbstractNacosRestTemplate { return execute(url, httpMethod, requestHttpEntity, responseType); } + /** + * Execute the HTTP method to the given URI template, writing the given request entity to the request, and returns + * the response as {@link HttpRestResult}. + * + * @param url url + * @param config HttpClientConfig + * @param header http header param + * @param query http query param + * @param body http body param + * @param httpMethod http method + * @param responseType return type + * @return {@link HttpRestResult} + * @throws Exception ex + */ + public HttpRestResult exchange(String url, HttpClientConfig config, Header header, Query query, + Object body, String httpMethod, Type responseType) throws Exception { + RequestHttpEntity requestHttpEntity = new RequestHttpEntity(config, header, query, body); + return execute(url, httpMethod, requestHttpEntity, responseType); + } + /** * Set the request interceptors that this accessor should use. * diff --git a/common/src/main/java/com/alibaba/nacos/common/http/client/request/DefaultHttpClientRequest.java b/common/src/main/java/com/alibaba/nacos/common/http/client/request/DefaultHttpClientRequest.java index 2a84c390b..d59010532 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/client/request/DefaultHttpClientRequest.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/client/request/DefaultHttpClientRequest.java @@ -65,7 +65,7 @@ public class DefaultHttpClientRequest implements HttpClientRequest { && requestHttpEntity.getBody() instanceof Map) { HttpUtils.initRequestFromEntity(httpRequestBase, (Map) requestHttpEntity.getBody(), headers.getCharset()); } else { - HttpUtils.initRequestEntity(httpRequestBase, requestHttpEntity.getBody(), headers.getValue(HttpHeaderConsts.CONTENT_TYPE)); + HttpUtils.initRequestEntity(httpRequestBase, requestHttpEntity.getBody(), headers); } replaceDefaultConfig(httpRequestBase, requestHttpEntity.getHttpClientConfig()); return httpRequestBase; diff --git a/common/src/main/java/com/alibaba/nacos/common/http/client/request/JdkHttpClientRequest.java b/common/src/main/java/com/alibaba/nacos/common/http/client/request/JdkHttpClientRequest.java index 7649b4eb5..fd921158c 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/client/request/JdkHttpClientRequest.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/client/request/JdkHttpClientRequest.java @@ -90,7 +90,7 @@ public class JdkHttpClientRequest implements HttpClientRequest { conn.setConnectTimeout(this.httpClientConfig.getConTimeOutMillis()); conn.setReadTimeout(this.httpClientConfig.getReadTimeOutMillis()); conn.setRequestMethod(httpMethod); - if (body != null) { + if (body != null && !"".equals(body)) { String contentType = headers.getValue(HttpHeaderConsts.CONTENT_TYPE); String bodyStr = JacksonUtils.toJson(body); if (MediaType.APPLICATION_FORM_URLENCODED.equals(contentType)) { diff --git a/common/src/main/java/com/alibaba/nacos/common/http/client/response/JdkHttpClientResponse.java b/common/src/main/java/com/alibaba/nacos/common/http/client/response/JdkHttpClientResponse.java index f4ad0794d..90fef3445 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/client/response/JdkHttpClientResponse.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/client/response/JdkHttpClientResponse.java @@ -16,9 +16,11 @@ package com.alibaba.nacos.common.http.client.response; +import com.alibaba.nacos.common.constant.HttpHeaderConsts; import com.alibaba.nacos.common.http.param.Header; import com.alibaba.nacos.common.utils.IoUtils; +import java.io.ByteArrayInputStream; import java.io.IOException; import java.io.InputStream; import java.net.HttpURLConnection; @@ -36,6 +38,8 @@ public class JdkHttpClientResponse implements HttpClientResponse { private Header responseHeader; + private static final String CONTENT_ENCODING = "gzip"; + public JdkHttpClientResponse(HttpURLConnection conn) { this.conn = conn; } @@ -51,8 +55,18 @@ public class JdkHttpClientResponse implements HttpClientResponse { @Override public InputStream getBody() throws IOException { + Header headers = getHeaders(); InputStream errorStream = this.conn.getErrorStream(); this.responseStream = (errorStream != null ? errorStream : this.conn.getInputStream()); + String contentEncoding = headers.getValue(HttpHeaderConsts.CONTENT_ENCODING); + // Used to process http content_encoding, when content_encoding is GZIP, use GZIPInputStream + if (CONTENT_ENCODING.equals(contentEncoding)) { + byte[] bytes = IoUtils.tryDecompress(this.responseStream); + if (bytes == null) { + throw new IOException("decompress http response error"); + } + return new ByteArrayInputStream(bytes); + } return this.responseStream; } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/param/Header.java b/common/src/main/java/com/alibaba/nacos/common/http/param/Header.java index bebba34ea..fcaf31c72 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/param/Header.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/param/Header.java @@ -18,6 +18,7 @@ package com.alibaba.nacos.common.http.param; import com.alibaba.nacos.api.common.Constants; import com.alibaba.nacos.common.constant.HttpHeaderConsts; +import com.alibaba.nacos.common.utils.MapUtils; import com.alibaba.nacos.common.utils.StringUtils; import java.util.ArrayList; @@ -45,7 +46,6 @@ public class Header { addParam(HttpHeaderConsts.CONTENT_TYPE, MediaType.APPLICATION_JSON); addParam(HttpHeaderConsts.ACCEPT_CHARSET, "UTF-8"); addParam(HttpHeaderConsts.ACCEPT_ENCODING, "gzip"); - addParam(HttpHeaderConsts.CONTENT_ENCODING, "gzip"); } public static Header newInstance() { @@ -130,8 +130,10 @@ public class Header { * @param params parameters */ public void addAll(Map params) { - for (Map.Entry entry : params.entrySet()) { - addParam(entry.getKey(), entry.getValue()); + if (MapUtils.isNotEmpty(params)) { + for (Map.Entry entry : params.entrySet()) { + addParam(entry.getKey(), entry.getValue()); + } } } @@ -143,9 +145,11 @@ public class Header { * @param headers original response header */ public void setOriginalResponseHeader(Map> headers) { - this.originalResponseHeader.putAll(headers); - for (Map.Entry> entry : this.originalResponseHeader.entrySet()) { - addParam(entry.getKey(), entry.getValue().get(0)); + if (MapUtils.isNotEmpty(headers)) { + this.originalResponseHeader.putAll(headers); + for (Map.Entry> entry : this.originalResponseHeader.entrySet()) { + addParam(entry.getKey(), entry.getValue().get(0)); + } } } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/param/MediaType.java b/common/src/main/java/com/alibaba/nacos/common/http/param/MediaType.java index ecb05abb1..774c48307 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/param/MediaType.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/param/MediaType.java @@ -16,6 +16,9 @@ package com.alibaba.nacos.common.http.param; +import com.alibaba.nacos.api.common.Constants; +import com.alibaba.nacos.common.utils.StringUtils; + /** * Http Media type. * @@ -23,12 +26,9 @@ package com.alibaba.nacos.common.http.param; */ public final class MediaType { - private MediaType() { - } - public static final String APPLICATION_ATOM_XML = "application/atom+xml"; - public static final String APPLICATION_FORM_URLENCODED = "application/x-www-form-urlencoded"; + public static final String APPLICATION_FORM_URLENCODED = "application/x-www-form-urlencoded;charset=UTF-8"; public static final String APPLICATION_OCTET_STREAM = "application/octet-stream"; @@ -36,14 +36,76 @@ public final class MediaType { public static final String APPLICATION_XHTML_XML = "application/xhtml+xml"; - public static final String APPLICATION_XML = "application/xml"; + public static final String APPLICATION_XML = "application/xml;charset=UTF-8"; - public static final String APPLICATION_JSON = "application/json"; + public static final String APPLICATION_JSON = "application/json;charset=UTF-8"; - public static final String MULTIPART_FORM_DATA = "multipart/form-data"; + public static final String MULTIPART_FORM_DATA = "multipart/form-data;charset=UTF-8"; - public static final String TEXT_HTML = "text/html"; + public static final String TEXT_HTML = "text/html;charset=UTF-8"; - public static final String TEXT_PLAIN = "text/plain"; + public static final String TEXT_PLAIN = "text/plain;charset=UTF-8"; + private MediaType(String type, String charset) { + this.type = type; + this.charset = charset; + } + + /** + * content type. + */ + private final String type; + + /** + * content type charset. + */ + private final String charset; + + /** + * Parse the given String contentType into a {@code MediaType} object. + * + * @param contentType mediaType + * @return MediaType + */ + public static MediaType valueOf(String contentType) { + if (StringUtils.isEmpty(contentType)) { + throw new IllegalArgumentException("MediaType must not be empty"); + } + String[] values = contentType.split(";"); + String charset = Constants.ENCODE; + for (String value : values) { + if (value.startsWith("charset=")) { + charset = value.substring("charset=".length()); + } + } + return new MediaType(values[0], charset); + } + + /** + * Use the given contentType and charset to assemble into a {@code MediaType} object. + * + * @param contentType contentType + * @param charset charset + * @return MediaType + */ + public static MediaType valueOf(String contentType, String charset) { + if (StringUtils.isEmpty(contentType)) { + throw new IllegalArgumentException("MediaType must not be empty"); + } + String[] values = contentType.split(";"); + return new MediaType(values[0], StringUtils.isEmpty(charset) ? Constants.ENCODE : charset); + } + + public String getType() { + return type; + } + + public String getCharset() { + return charset; + } + + @Override + public String toString() { + return type + ";charset=" + charset; + } } diff --git a/common/src/main/java/com/alibaba/nacos/common/http/param/Query.java b/common/src/main/java/com/alibaba/nacos/common/http/param/Query.java index 1b2413343..4c4029e42 100644 --- a/common/src/main/java/com/alibaba/nacos/common/http/param/Query.java +++ b/common/src/main/java/com/alibaba/nacos/common/http/param/Query.java @@ -16,6 +16,8 @@ package com.alibaba.nacos.common.http.param; +import com.alibaba.nacos.common.utils.MapUtils; + import java.io.UnsupportedEncodingException; import java.net.URLEncoder; import java.util.LinkedHashMap; @@ -68,8 +70,10 @@ public class Query { * @return this query */ public Query initParams(Map params) { - for (Map.Entry entry : params.entrySet()) { - addParam(entry.getKey(), entry.getValue()); + if (MapUtils.isNotEmpty(params)) { + for (Map.Entry entry : params.entrySet()) { + addParam(entry.getKey(), entry.getValue()); + } } return this; } diff --git a/common/src/main/java/com/alibaba/nacos/common/model/RequestHttpEntity.java b/common/src/main/java/com/alibaba/nacos/common/model/RequestHttpEntity.java index b2c03253d..2240002e6 100644 --- a/common/src/main/java/com/alibaba/nacos/common/model/RequestHttpEntity.java +++ b/common/src/main/java/com/alibaba/nacos/common/model/RequestHttpEntity.java @@ -26,7 +26,6 @@ import java.util.Map; * Represents an HTTP request , consisting of headers and body. * * @author mai.jh - * @date 2020/5/23 */ public class RequestHttpEntity { @@ -36,7 +35,7 @@ public class RequestHttpEntity { private final Query query; - private Object body; + private final Object body; public RequestHttpEntity(Header header, Query query) { this(null, header, query); diff --git a/common/src/main/java/com/alibaba/nacos/common/notify/DefaultSharePublisher.java b/common/src/main/java/com/alibaba/nacos/common/notify/DefaultSharePublisher.java index 3028423f7..a2fa996d1 100644 --- a/common/src/main/java/com/alibaba/nacos/common/notify/DefaultSharePublisher.java +++ b/common/src/main/java/com/alibaba/nacos/common/notify/DefaultSharePublisher.java @@ -96,6 +96,10 @@ public class DefaultSharePublisher extends DefaultPublisher { // Get for Map, the algorithm is O(1). Set subscribers = subMappings.get(slowEventType); + if (null == subscribers) { + LOGGER.debug("[NotifyCenter] No subscribers for slow event {}", slowEventType.getName()); + return; + } // Notification single event subscriber for (Subscriber subscriber : subscribers) { diff --git a/common/src/main/java/com/alibaba/nacos/common/utils/ConvertUtils.java b/common/src/main/java/com/alibaba/nacos/common/utils/ConvertUtils.java index 14432eced..50670e411 100644 --- a/common/src/main/java/com/alibaba/nacos/common/utils/ConvertUtils.java +++ b/common/src/main/java/com/alibaba/nacos/common/utils/ConvertUtils.java @@ -51,7 +51,11 @@ public final class ConvertUtils { if (StringUtils.isBlank(val)) { return defaultValue; } - return Integer.parseInt(val); + try { + return Integer.parseInt(val); + } catch (NumberFormatException exception) { + return defaultValue; + } } /** @@ -77,7 +81,11 @@ public final class ConvertUtils { if (StringUtils.isBlank(val)) { return defaultValue; } - return Long.parseLong(val); + try { + return Long.parseLong(val); + } catch (NumberFormatException exception) { + return defaultValue; + } } /** @@ -92,7 +100,11 @@ public final class ConvertUtils { if (StringUtils.isBlank(val)) { return defaultValue; } - return Boolean.parseBoolean(val); + try { + return Boolean.parseBoolean(val); + } catch (NumberFormatException exception) { + return defaultValue; + } } // The following utility functions are extracted from org.apache.commons.lang3 @@ -126,7 +138,7 @@ public final class ConvertUtils { * @return the boolean value of the string, {@code false} if no match or the String is null */ public static boolean toBoolean(final String str) { - return toBooleanObject(str) == Boolean.TRUE; + return Boolean.TRUE.equals(toBooleanObject(str)); } /** diff --git a/common/src/main/java/com/alibaba/nacos/common/utils/HttpMethod.java b/common/src/main/java/com/alibaba/nacos/common/utils/HttpMethod.java index 688433fac..c605bd9b9 100644 --- a/common/src/main/java/com/alibaba/nacos/common/utils/HttpMethod.java +++ b/common/src/main/java/com/alibaba/nacos/common/utils/HttpMethod.java @@ -26,8 +26,10 @@ public class HttpMethod { public static final String GET = "GET"; - // this is only use in nacos, Custom request type, essentially a get request - + /** + * this is only use in nacos, Custom request type, essentially a GET request, Mainly used for GET request parameters + * are relatively large,can not be placed on the URL, so it needs to be placed in the body. + */ public static final String GET_LARGE = "GET-LARGE"; public static final String HEAD = "HEAD"; @@ -40,6 +42,12 @@ public class HttpMethod { public static final String DELETE = "DELETE"; + /** + * this is only use in nacos, Custom request type, essentially a DELETE request, Mainly used for DELETE request + * parameters are relatively large, can not be placed on the URL, so it needs to be placed in the body. + */ + public static final String DELETE_LARGE = "DELETE_LARGE"; + public static final String OPTIONS = "OPTIONS"; public static final String TRACE = "TRACE"; diff --git a/common/src/main/java/com/alibaba/nacos/common/utils/IoUtils.java b/common/src/main/java/com/alibaba/nacos/common/utils/IoUtils.java index 0f949ba42..c38de6dad 100644 --- a/common/src/main/java/com/alibaba/nacos/common/utils/IoUtils.java +++ b/common/src/main/java/com/alibaba/nacos/common/utils/IoUtils.java @@ -50,9 +50,9 @@ public class IoUtils { * * @param raw compress stream * @return byte array after decompress - * @throws Exception exception + * @throws IOException exception */ - public static byte[] tryDecompress(InputStream raw) throws Exception { + public static byte[] tryDecompress(InputStream raw) throws IOException { GZIPInputStream gis = null; ByteArrayOutputStream out = null; try { @@ -60,7 +60,7 @@ public class IoUtils { out = new ByteArrayOutputStream(); copy(gis, out); return out.toByteArray(); - } catch (Exception e) { + } catch (IOException e) { e.printStackTrace(); } finally { if (out != null) { diff --git a/common/src/test/java/com/alibaba/nacos/common/http/param/MediaTypeTest.java b/common/src/test/java/com/alibaba/nacos/common/http/param/MediaTypeTest.java new file mode 100644 index 000000000..eddbfecb3 --- /dev/null +++ b/common/src/test/java/com/alibaba/nacos/common/http/param/MediaTypeTest.java @@ -0,0 +1,62 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.common.http.param; + +import org.junit.Test; + +import static org.junit.Assert.assertEquals; + +/** + * MediaTypeTest. + * + * @author mai.jh + */ +public class MediaTypeTest { + + @Test + public void testValueOf() { + MediaType mediaType = MediaType.valueOf(MediaType.APPLICATION_FORM_URLENCODED); + String type = "application/x-www-form-urlencoded"; + String charset = "UTF-8"; + assertEquals(type, mediaType.getType()); + assertEquals(charset, mediaType.getCharset()); + assertEquals(MediaType.APPLICATION_FORM_URLENCODED, mediaType.toString()); + } + + @Test + public void testValueOf2() { + MediaType mediaType = MediaType.valueOf(MediaType.APPLICATION_FORM_URLENCODED, "ISO-8859-1"); + String type = "application/x-www-form-urlencoded"; + String charset = "ISO-8859-1"; + String excepted = "application/x-www-form-urlencoded;charset=ISO-8859-1"; + assertEquals(type, mediaType.getType()); + assertEquals(charset, mediaType.getCharset()); + assertEquals(excepted, mediaType.toString()); + } + + @Test + public void testValueOf3() { + MediaType mediaType = MediaType.valueOf("application/x-www-form-urlencoded", "ISO-8859-1"); + String type = "application/x-www-form-urlencoded"; + String charset = "ISO-8859-1"; + String excepted = "application/x-www-form-urlencoded;charset=ISO-8859-1"; + assertEquals(type, mediaType.getType()); + assertEquals(charset, mediaType.getCharset()); + assertEquals(excepted, mediaType.toString()); + } + +} diff --git a/common/src/test/java/com/alibaba/nacos/common/notify/NotifyCenterTest.java b/common/src/test/java/com/alibaba/nacos/common/notify/NotifyCenterTest.java index a7163938d..d1abd7c1e 100644 --- a/common/src/test/java/com/alibaba/nacos/common/notify/NotifyCenterTest.java +++ b/common/src/test/java/com/alibaba/nacos/common/notify/NotifyCenterTest.java @@ -132,6 +132,7 @@ public class NotifyCenterTest { return ExpireEvent.class; } + @Override public boolean ignoreExpireEvent() { return true; } diff --git a/config/src/main/java/com/alibaba/nacos/config/server/configuration/ConditionDistributedEmbedStorage.java b/config/src/main/java/com/alibaba/nacos/config/server/configuration/ConditionDistributedEmbedStorage.java index 176b1162e..5d3d0b8a6 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/configuration/ConditionDistributedEmbedStorage.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/configuration/ConditionDistributedEmbedStorage.java @@ -17,7 +17,7 @@ package com.alibaba.nacos.config.server.configuration; import com.alibaba.nacos.config.server.utils.PropertyUtil; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.springframework.context.annotation.Condition; import org.springframework.context.annotation.ConditionContext; import org.springframework.core.type.AnnotatedTypeMetadata; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/configuration/ConditionStandaloneEmbedStorage.java b/config/src/main/java/com/alibaba/nacos/config/server/configuration/ConditionStandaloneEmbedStorage.java index 4380d3811..2a6fcf56d 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/configuration/ConditionStandaloneEmbedStorage.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/configuration/ConditionStandaloneEmbedStorage.java @@ -17,7 +17,7 @@ package com.alibaba.nacos.config.server.configuration; import com.alibaba.nacos.config.server.utils.PropertyUtil; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.springframework.context.annotation.Condition; import org.springframework.context.annotation.ConditionContext; import org.springframework.core.type.AnnotatedTypeMetadata; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/controller/ConfigController.java b/config/src/main/java/com/alibaba/nacos/config/server/controller/ConfigController.java index be8505eed..95c775d05 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/controller/ConfigController.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/controller/ConfigController.java @@ -43,9 +43,10 @@ import com.alibaba.nacos.config.server.service.trace.ConfigTraceService; import com.alibaba.nacos.config.server.utils.MD5Util; import com.alibaba.nacos.config.server.utils.ParamUtils; import com.alibaba.nacos.config.server.utils.RequestUtil; +import com.alibaba.nacos.config.server.utils.NamespaceUtil; import com.alibaba.nacos.config.server.utils.TimeUtils; import com.alibaba.nacos.config.server.utils.ZipUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.time.DateFormatUtils; import org.slf4j.Logger; @@ -90,8 +91,6 @@ public class ConfigController { private static final Logger LOGGER = LoggerFactory.getLogger(ConfigController.class); - private static final String NAMESPACE_PUBLIC_KEY = "public"; - private static final String EXPORT_CONFIG_FILE_NAME = "nacos_config_export_"; private static final String EXPORT_CONFIG_FILE_NAME_EXT = ".zip"; @@ -175,7 +174,7 @@ public class ConfigController { .notifyConfigChange(new ConfigDataChangeEvent(true, dataId, group, tenant, time.getTime())); } ConfigTraceService - .logPersistenceEvent(dataId, group, tenant, requestIpApp, time.getTime(), InetUtils.getSelfIp(), + .logPersistenceEvent(dataId, group, tenant, requestIpApp, time.getTime(), InetUtils.getSelfIP(), ConfigTraceService.PERSISTENCE_EVENT_PUB, content); return true; } @@ -196,7 +195,7 @@ public class ConfigController { throws IOException, ServletException, NacosException { // check tenant ParamUtils.checkTenant(tenant); - tenant = processTenant(tenant); + tenant = NamespaceUtil.processNamespaceParameter(tenant); // check params ParamUtils.checkParam(dataId, group, "datumId", "content"); ParamUtils.checkParam(tag); @@ -469,7 +468,7 @@ public class ConfigController { @RequestParam(value = "tenant", required = false, defaultValue = StringUtils.EMPTY) String tenant, @RequestParam(value = "ids", required = false) List ids) { ids.removeAll(Collections.singleton(null)); - tenant = processTenant(tenant); + tenant = NamespaceUtil.processNamespaceParameter(tenant); List dataList = persistService.findAllConfigInfo4Export(dataId, group, tenant, appName, ids); List zipItemList = new ArrayList<>(); StringBuilder metaData = null; @@ -527,12 +526,12 @@ public class ConfigController { return ResultBuilder.buildResult(ResultCodeEnum.DATA_EMPTY, failedData); } - if (StringUtils.isNotBlank(namespace)) { - if (persistService.tenantInfoCountByTenantId(namespace) <= 0) { - failedData.put("succCount", 0); - return ResultBuilder.buildResult(ResultCodeEnum.NAMESPACE_NOT_EXIST, failedData); - } + namespace = NamespaceUtil.processNamespaceParameter(namespace); + if (StringUtils.isNotBlank(namespace) && persistService.tenantInfoCountByTenantId(namespace) <= 0) { + failedData.put("succCount", 0); + return ResultBuilder.buildResult(ResultCodeEnum.NAMESPACE_NOT_EXIST, failedData); } + List configInfoList = null; try { ZipUtils.UnZipResult unziped = ZipUtils.unzip(file.getBytes()); @@ -598,7 +597,7 @@ public class ConfigController { configInfo.getTenant(), time.getTime())); ConfigTraceService .logPersistenceEvent(configInfo.getDataId(), configInfo.getGroup(), configInfo.getTenant(), - requestIpApp, time.getTime(), InetUtils.getSelfIp(), + requestIpApp, time.getTime(), InetUtils.getSelfIP(), ConfigTraceService.PERSISTENCE_EVENT_PUB, configInfo.getContent()); } return ResultBuilder.buildSuccessResult("导入成功", saveResult); @@ -628,10 +627,9 @@ public class ConfigController { return ResultBuilder.buildResult(ResultCodeEnum.NO_SELECTED_CONFIG, failedData); } configBeansList.removeAll(Collections.singleton(null)); - - if (NAMESPACE_PUBLIC_KEY.equalsIgnoreCase(namespace)) { - namespace = ""; - } else if (persistService.tenantInfoCountByTenantId(namespace) <= 0) { + + namespace = NamespaceUtil.processNamespaceParameter(namespace); + if (StringUtils.isNotBlank(namespace) && persistService.tenantInfoCountByTenantId(namespace) <= 0) { failedData.put("succCount", 0); return ResultBuilder.buildResult(ResultCodeEnum.NAMESPACE_NOT_EXIST, failedData); } @@ -684,17 +682,10 @@ public class ConfigController { configInfo.getTenant(), time.getTime())); ConfigTraceService .logPersistenceEvent(configInfo.getDataId(), configInfo.getGroup(), configInfo.getTenant(), - requestIpApp, time.getTime(), InetUtils.getSelfIp(), + requestIpApp, time.getTime(), InetUtils.getSelfIP(), ConfigTraceService.PERSISTENCE_EVENT_PUB, configInfo.getContent()); } return ResultBuilder.buildSuccessResult("Clone Completed Successfully", saveResult); } - private String processTenant(String tenant) { - if (StringUtils.isEmpty(tenant) || NAMESPACE_PUBLIC_KEY.equalsIgnoreCase(tenant)) { - return ""; - } - return tenant; - } - } diff --git a/config/src/main/java/com/alibaba/nacos/config/server/controller/ConfigOpsController.java b/config/src/main/java/com/alibaba/nacos/config/server/controller/ConfigOpsController.java index 54bf19c23..6c94e1d98 100755 --- a/config/src/main/java/com/alibaba/nacos/config/server/controller/ConfigOpsController.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/controller/ConfigOpsController.java @@ -31,7 +31,7 @@ import com.alibaba.nacos.config.server.service.repository.PersistService; import com.alibaba.nacos.config.server.service.repository.embedded.DatabaseOperate; import com.alibaba.nacos.config.server.utils.LogUtil; import com.alibaba.nacos.config.server.utils.PropertyUtil; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.WebUtils; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/controller/HealthController.java b/config/src/main/java/com/alibaba/nacos/config/server/controller/HealthController.java index 1be5375c0..1fa35f643 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/controller/HealthController.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/controller/HealthController.java @@ -20,7 +20,7 @@ import com.alibaba.nacos.config.server.constant.Constants; import com.alibaba.nacos.config.server.service.datasource.DataSourceService; import com.alibaba.nacos.config.server.service.datasource.DynamicDataSource; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RequestMapping; @@ -69,7 +69,7 @@ public class HealthController { sb.append("master db (").append(dbStatus.split(":")[1]).append(") down. "); } if (!memberManager.isInIpList()) { - sb.append("server ip ").append(InetUtils.getSelfIp()) + sb.append("server ip ").append(InetUtils.getSelfIP()) .append(" is not in the serverList of address server. "); } } diff --git a/config/src/main/java/com/alibaba/nacos/config/server/model/app/ApplicationInfo.java b/config/src/main/java/com/alibaba/nacos/config/server/model/app/ApplicationInfo.java index 48930e5a1..acc318d7b 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/model/app/ApplicationInfo.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/model/app/ApplicationInfo.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.config.server.model.app; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.InetUtils; /** * ApplicationInfo. @@ -102,7 +102,7 @@ public class ApplicationInfo { */ public boolean canCurrentServerOwnTheLock() { boolean currentOwnerIsMe = - subInfoCollectLockOwner == null || InetUtils.getSelfIp().equals(subInfoCollectLockOwner); + subInfoCollectLockOwner == null || InetUtils.getSelfIP().equals(subInfoCollectLockOwner); if (currentOwnerIsMe) { return true; @@ -115,7 +115,7 @@ public class ApplicationInfo { } public String currentServer() { - return InetUtils.getSelfIp(); + return InetUtils.getSelfIP(); } } diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/ConfigChangePublisher.java b/config/src/main/java/com/alibaba/nacos/config/server/service/ConfigChangePublisher.java index 99fdeaf0b..928641657 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/ConfigChangePublisher.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/ConfigChangePublisher.java @@ -19,7 +19,7 @@ package com.alibaba.nacos.config.server.service; import com.alibaba.nacos.common.notify.NotifyCenter; import com.alibaba.nacos.config.server.model.event.ConfigDataChangeEvent; import com.alibaba.nacos.config.server.utils.PropertyUtil; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; /** * ConfigChangePublisher. diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/ConfigSubService.java b/config/src/main/java/com/alibaba/nacos/config/server/service/ConfigSubService.java index 157f373a5..03b6bc8cb 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/ConfigSubService.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/ConfigSubService.java @@ -25,7 +25,7 @@ import com.alibaba.nacos.config.server.utils.ConfigExecutor; import com.alibaba.nacos.config.server.utils.LogUtil; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.lang3.StringUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/datasource/ExternalDataSourceServiceImpl.java b/config/src/main/java/com/alibaba/nacos/config/server/service/datasource/ExternalDataSourceServiceImpl.java index 725101282..58184d3df 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/datasource/ExternalDataSourceServiceImpl.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/datasource/ExternalDataSourceServiceImpl.java @@ -42,7 +42,7 @@ import com.alibaba.nacos.common.utils.StringUtils; import com.alibaba.nacos.config.server.monitor.MetricsMonitor; import com.alibaba.nacos.config.server.utils.ConfigExecutor; import com.alibaba.nacos.config.server.utils.PropertyUtil; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.zaxxer.hikari.HikariDataSource; /** diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/datasource/LocalDataSourceServiceImpl.java b/config/src/main/java/com/alibaba/nacos/config/server/service/datasource/LocalDataSourceServiceImpl.java index 068e50679..3fb412e4d 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/datasource/LocalDataSourceServiceImpl.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/datasource/LocalDataSourceServiceImpl.java @@ -21,8 +21,8 @@ import com.alibaba.nacos.api.exception.runtime.NacosRuntimeException; import com.alibaba.nacos.config.server.constant.Constants; import com.alibaba.nacos.config.server.utils.LogUtil; import com.alibaba.nacos.config.server.utils.PropertyUtil; -import com.alibaba.nacos.core.utils.DiskUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; import com.zaxxer.hikari.HikariDataSource; import java.io.File; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/dump/DumpService.java b/config/src/main/java/com/alibaba/nacos/config/server/service/dump/DumpService.java index a0ebf570c..4470269db 100755 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/dump/DumpService.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/dump/DumpService.java @@ -49,8 +49,8 @@ import com.alibaba.nacos.config.server.utils.GroupKey2; import com.alibaba.nacos.config.server.utils.LogUtil; import com.alibaba.nacos.config.server.utils.TimeUtils; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import com.alibaba.nacos.core.utils.TimerContext; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -419,7 +419,7 @@ public abstract class DumpService { } } else { // remove config info - persistService.removeConfigInfo(dataId, group, tenant, InetUtils.getSelfIp(), null); + persistService.removeConfigInfo(dataId, group, tenant, InetUtils.getSelfIP(), null); LOGGER.warn( "[merge-delete] delete config info because no datum. dataId=" + dataId + ", groupId=" + group); diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/dump/EmbeddedDumpService.java b/config/src/main/java/com/alibaba/nacos/config/server/service/dump/EmbeddedDumpService.java index f3d645ae1..85abd81fa 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/dump/EmbeddedDumpService.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/dump/EmbeddedDumpService.java @@ -28,7 +28,7 @@ import com.alibaba.nacos.consistency.cp.CPProtocol; import com.alibaba.nacos.consistency.cp.MetadataKey; import com.alibaba.nacos.core.cluster.ServerMemberManager; import com.alibaba.nacos.core.distributed.ProtocolManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.GlobalExecutor; import org.springframework.context.annotation.Conditional; import org.springframework.stereotype.Component; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/merge/MergeDatumService.java b/config/src/main/java/com/alibaba/nacos/config/server/service/merge/MergeDatumService.java index 61daddd3e..e08a56020 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/merge/MergeDatumService.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/merge/MergeDatumService.java @@ -27,8 +27,8 @@ import com.alibaba.nacos.config.server.utils.ContentUtils; import com.alibaba.nacos.config.server.utils.PropertyUtil; import com.alibaba.nacos.config.server.utils.TimeUtils; import com.alibaba.nacos.core.distributed.ProtocolManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; @@ -106,7 +106,7 @@ public class MergeDatumService { return; } for (ConfigInfoChanged item : persistService.findAllAggrGroup()) { - addMergeTask(item.getDataId(), item.getGroup(), item.getTenant(), InetUtils.getSelfIp()); + addMergeTask(item.getDataId(), item.getGroup(), item.getTenant(), InetUtils.getSelfIP()); } } @@ -163,7 +163,7 @@ public class MergeDatumService { ContentUtils.truncateContent(cf.getContent())); } else { // remove - persistService.removeConfigInfo(dataId, group, tenant, InetUtils.getSelfIp(), null); + persistService.removeConfigInfo(dataId, group, tenant, InetUtils.getSelfIP(), null); LOGGER.warn("[merge-delete] delete config info because no datum. dataId=" + dataId + ", groupId=" + group); } diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/merge/MergeTaskProcessor.java b/config/src/main/java/com/alibaba/nacos/config/server/service/merge/MergeTaskProcessor.java index 44a7a3085..f2f14808b 100755 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/merge/MergeTaskProcessor.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/merge/MergeTaskProcessor.java @@ -28,7 +28,7 @@ import com.alibaba.nacos.config.server.service.repository.PersistService; import com.alibaba.nacos.config.server.service.trace.ConfigTraceService; import com.alibaba.nacos.config.server.utils.ContentUtils; import com.alibaba.nacos.config.server.utils.TimeUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -84,7 +84,7 @@ public class MergeTaskProcessor implements NacosTaskProcessor { ContentUtils.truncateContent(cf.getContent())); ConfigTraceService - .logPersistenceEvent(dataId, group, tenant, null, time.getTime(), InetUtils.getSelfIp(), + .logPersistenceEvent(dataId, group, tenant, null, time.getTime(), InetUtils.getSelfIP(), ConfigTraceService.PERSISTENCE_EVENT_MERGE, cf.getContent()); } else { // remove @@ -98,7 +98,7 @@ public class MergeTaskProcessor implements NacosTaskProcessor { "[merge-delete] delete config info because no datum. dataId=" + dataId + ", groupId=" + group); ConfigTraceService - .logPersistenceEvent(dataId, group, tenant, null, time.getTime(), InetUtils.getSelfIp(), + .logPersistenceEvent(dataId, group, tenant, null, time.getTime(), InetUtils.getSelfIP(), ConfigTraceService.PERSISTENCE_EVENT_REMOVE, null); } NotifyCenter.publishEvent(new ConfigDataChangeEvent(false, dataId, group, tenant, tag, time.getTime())); diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/notify/AsyncNotifyService.java b/config/src/main/java/com/alibaba/nacos/config/server/service/notify/AsyncNotifyService.java index f38774bf5..1f26e6afe 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/notify/AsyncNotifyService.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/notify/AsyncNotifyService.java @@ -32,8 +32,8 @@ import com.alibaba.nacos.config.server.utils.ConfigExecutor; import com.alibaba.nacos.config.server.utils.LogUtil; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -127,14 +127,14 @@ public class AsyncNotifyService { if (unHealthNeedDelay) { // target ip is unhealthy, then put it in the notification list ConfigTraceService.logNotifyEvent(task.getDataId(), task.getGroup(), task.getTenant(), null, - task.getLastModified(), InetUtils.getSelfIp(), ConfigTraceService.NOTIFY_EVENT_UNHEALTH, + task.getLastModified(), InetUtils.getSelfIP(), ConfigTraceService.NOTIFY_EVENT_UNHEALTH, 0, task.target); // get delay time and set fail count to the task asyncTaskExecute(task); } else { Header header = Header.newInstance(); header.addParam(NotifyService.NOTIFY_HEADER_LAST_MODIFIED, String.valueOf(task.getLastModified())); - header.addParam(NotifyService.NOTIFY_HEADER_OP_HANDLE_IP, InetUtils.getSelfIp()); + header.addParam(NotifyService.NOTIFY_HEADER_OP_HANDLE_IP, InetUtils.getSelfIP()); if (task.isBeta) { header.addParam("isBeta", "true"); } @@ -168,13 +168,13 @@ public class AsyncNotifyService { if (result.ok()) { ConfigTraceService.logNotifyEvent(task.getDataId(), task.getGroup(), task.getTenant(), null, - task.getLastModified(), InetUtils.getSelfIp(), ConfigTraceService.NOTIFY_EVENT_OK, delayed, + task.getLastModified(), InetUtils.getSelfIP(), ConfigTraceService.NOTIFY_EVENT_OK, delayed, task.target); } else { LOGGER.error("[notify-error] target:{} dataId:{} group:{} ts:{} code:{}", task.target, task.getDataId(), task.getGroup(), task.getLastModified(), result.getCode()); ConfigTraceService.logNotifyEvent(task.getDataId(), task.getGroup(), task.getTenant(), null, - task.getLastModified(), InetUtils.getSelfIp(), ConfigTraceService.NOTIFY_EVENT_ERROR, delayed, + task.getLastModified(), InetUtils.getSelfIP(), ConfigTraceService.NOTIFY_EVENT_ERROR, delayed, task.target); //get delay time and set fail count to the task @@ -196,7 +196,7 @@ public class AsyncNotifyService { task.getGroup(), task.getLastModified(), ex.toString()); ConfigTraceService .logNotifyEvent(task.getDataId(), task.getGroup(), task.getTenant(), null, task.getLastModified(), - InetUtils.getSelfIp(), ConfigTraceService.NOTIFY_EVENT_EXCEPTION, delayed, task.target); + InetUtils.getSelfIP(), ConfigTraceService.NOTIFY_EVENT_EXCEPTION, delayed, task.target); //get delay time and set fail count to the task asyncTaskExecute(task); diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/notify/NotifyTaskProcessor.java b/config/src/main/java/com/alibaba/nacos/config/server/service/notify/NotifyTaskProcessor.java index 9002f0eb8..02282693f 100755 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/notify/NotifyTaskProcessor.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/notify/NotifyTaskProcessor.java @@ -24,8 +24,8 @@ import com.alibaba.nacos.config.server.monitor.MetricsMonitor; import com.alibaba.nacos.config.server.service.trace.ConfigTraceService; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -73,13 +73,13 @@ public class NotifyTaskProcessor implements NacosTaskProcessor { */ List headers = Arrays .asList(NotifyService.NOTIFY_HEADER_LAST_MODIFIED, String.valueOf(lastModified), - NotifyService.NOTIFY_HEADER_OP_HANDLE_IP, InetUtils.getSelfIp()); + NotifyService.NOTIFY_HEADER_OP_HANDLE_IP, InetUtils.getSelfIP()); String urlString = MessageFormat .format(URL_PATTERN, serverIp, ApplicationUtils.getContextPath(), dataId, group); RestResult result = NotifyService.invokeURL(urlString, headers, Constants.ENCODE); if (result.ok()) { - ConfigTraceService.logNotifyEvent(dataId, group, tenant, null, lastModified, InetUtils.getSelfIp(), + ConfigTraceService.logNotifyEvent(dataId, group, tenant, null, lastModified, InetUtils.getSelfIP(), ConfigTraceService.NOTIFY_EVENT_OK, delayed, serverIp); MetricsMonitor.getNotifyRtTimer().record(delayed, TimeUnit.MILLISECONDS); @@ -89,7 +89,7 @@ public class NotifyTaskProcessor implements NacosTaskProcessor { MetricsMonitor.getConfigNotifyException().increment(); LOGGER.error("[notify-error] {}, {}, to {}, result {}", new Object[] {dataId, group, serverIp, result.getCode()}); - ConfigTraceService.logNotifyEvent(dataId, group, tenant, null, lastModified, InetUtils.getSelfIp(), + ConfigTraceService.logNotifyEvent(dataId, group, tenant, null, lastModified, InetUtils.getSelfIP(), ConfigTraceService.NOTIFY_EVENT_ERROR, delayed, serverIp); return false; } @@ -97,7 +97,7 @@ public class NotifyTaskProcessor implements NacosTaskProcessor { MetricsMonitor.getConfigNotifyException().increment(); LOGGER.error("[notify-exception] " + dataId + ", " + group + ", to " + serverIp + ", " + e.toString()); LOGGER.debug("[notify-exception] " + dataId + ", " + group + ", to " + serverIp + ", " + e.toString(), e); - ConfigTraceService.logNotifyEvent(dataId, group, tenant, null, lastModified, InetUtils.getSelfIp(), + ConfigTraceService.logNotifyEvent(dataId, group, tenant, null, lastModified, InetUtils.getSelfIP(), ConfigTraceService.NOTIFY_EVENT_EXCEPTION, delayed, serverIp); return false; } diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/DerbySnapshotOperation.java b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/DerbySnapshotOperation.java index 8fa6c7b10..7357c393d 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/DerbySnapshotOperation.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/DerbySnapshotOperation.java @@ -27,8 +27,8 @@ import com.alibaba.nacos.consistency.snapshot.Reader; import com.alibaba.nacos.consistency.snapshot.SnapshotOperation; import com.alibaba.nacos.consistency.snapshot.Writer; import com.alibaba.nacos.core.distributed.raft.utils.RaftExecutor; -import com.alibaba.nacos.core.utils.DiskUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.TimerContext; import com.alipay.sofa.jraft.util.CRC64; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/DistributedDatabaseOperateImpl.java b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/DistributedDatabaseOperateImpl.java index 081a7e5dd..cfd0a7b19 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/DistributedDatabaseOperateImpl.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/DistributedDatabaseOperateImpl.java @@ -57,7 +57,7 @@ import com.alibaba.nacos.consistency.snapshot.SnapshotOperation; import com.alibaba.nacos.core.cluster.ServerMemberManager; import com.alibaba.nacos.core.distributed.ProtocolManager; import com.alibaba.nacos.core.utils.ClassUtils; -import com.alibaba.nacos.core.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; import com.alibaba.nacos.core.utils.GenericType; import com.google.common.base.Preconditions; import com.google.protobuf.ByteString; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/StandaloneDatabaseOperateImpl.java b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/StandaloneDatabaseOperateImpl.java index 332494fb0..d556e9d3d 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/StandaloneDatabaseOperateImpl.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/embedded/StandaloneDatabaseOperateImpl.java @@ -24,7 +24,7 @@ import com.alibaba.nacos.config.server.service.datasource.DataSourceService; import com.alibaba.nacos.config.server.service.datasource.DynamicDataSource; import com.alibaba.nacos.config.server.service.sql.ModifyRequest; import com.alibaba.nacos.config.server.utils.LogUtil; -import com.alibaba.nacos.core.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; import org.apache.commons.lang3.BooleanUtils; import org.springframework.context.annotation.Conditional; import org.springframework.jdbc.core.JdbcTemplate; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/extrnal/ExternalStoragePaginationHelperImpl.java b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/extrnal/ExternalStoragePaginationHelperImpl.java index 96e1311e0..18323134e 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/extrnal/ExternalStoragePaginationHelperImpl.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/extrnal/ExternalStoragePaginationHelperImpl.java @@ -20,7 +20,7 @@ import com.alibaba.nacos.config.server.model.Page; import com.alibaba.nacos.config.server.service.repository.PaginationHelper; import com.alibaba.nacos.config.server.service.sql.EmbeddedStorageContextUtils; import com.alibaba.nacos.config.server.utils.PropertyUtil; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.springframework.jdbc.core.JdbcTemplate; import org.springframework.jdbc.core.RowMapper; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/extrnal/ExternalStoragePersistServiceImpl.java b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/extrnal/ExternalStoragePersistServiceImpl.java index 22c9a6ea7..276b61afe 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/repository/extrnal/ExternalStoragePersistServiceImpl.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/repository/extrnal/ExternalStoragePersistServiceImpl.java @@ -2292,7 +2292,7 @@ public class ExternalStoragePersistServiceImpl implements PersistService { .queryForObject(sqlFetchRows, new Object[] {nid}, HISTORY_DETAIL_ROW_MAPPER); return historyInfo; } catch (DataAccessException e) { - LogUtil.FATAL_LOG.error("[list-config-history] error, nid:{}", new Object[] {nid}, e); + LogUtil.FATAL_LOG.error("[detail-config-history] error, nid:{}", new Object[] {nid}, e); throw e; } } diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/sql/EmbeddedStorageContextUtils.java b/config/src/main/java/com/alibaba/nacos/config/server/service/sql/EmbeddedStorageContextUtils.java index 3a4f31372..8fe679396 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/sql/EmbeddedStorageContextUtils.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/sql/EmbeddedStorageContextUtils.java @@ -21,7 +21,7 @@ import com.alibaba.nacos.common.utils.StringUtils; import com.alibaba.nacos.config.server.constant.Constants; import com.alibaba.nacos.config.server.model.ConfigInfo; import com.alibaba.nacos.config.server.model.event.ConfigDumpEvent; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import java.sql.Timestamp; import java.util.ArrayList; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/service/trace/ConfigTraceService.java b/config/src/main/java/com/alibaba/nacos/config/server/service/trace/ConfigTraceService.java index de92852b7..e36f82f4b 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/service/trace/ConfigTraceService.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/service/trace/ConfigTraceService.java @@ -20,7 +20,7 @@ import com.alibaba.nacos.common.utils.MD5Utils; import com.alibaba.nacos.config.server.constant.Constants; import com.alibaba.nacos.config.server.monitor.MetricsMonitor; import com.alibaba.nacos.config.server.utils.LogUtil; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.apache.commons.lang3.StringUtils; import org.springframework.stereotype.Service; @@ -87,7 +87,7 @@ public class ConfigTraceService { // (md5) String md5 = content == null ? null : MD5Utils.md5Hex(content, Constants.ENCODE); - LogUtil.TRACE_LOG.info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIp(), dataId, group, tenant, + LogUtil.TRACE_LOG.info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIP(), dataId, group, tenant, requestIpAppName, ts, handleIp, "persist", type, -1, md5); } @@ -116,7 +116,7 @@ public class ConfigTraceService { } //localIp | dataid | group | tenant | requestIpAppName | ts | handleIp | event | type | [delayed] | ext // (targetIp) - LogUtil.TRACE_LOG.info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIp(), dataId, group, tenant, + LogUtil.TRACE_LOG.info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIP(), dataId, group, tenant, requestIpAppName, ts, handleIp, "notify", type, delayed, targetIp); } @@ -143,7 +143,7 @@ public class ConfigTraceService { tenant = null; } //localIp | dataid | group | tenant | requestIpAppName | ts | handleIp | event | type | [delayed] | length - LogUtil.TRACE_LOG.info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIp(), dataId, group, tenant, + LogUtil.TRACE_LOG.info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIP(), dataId, group, tenant, requestIpAppName, ts, handleIp, "dump", type, delayed, length); } @@ -169,7 +169,7 @@ public class ConfigTraceService { } //localIp | dataid | group | tenant | requestIpAppName | ts | handleIp | event | type | [delayed = -1] LogUtil.TRACE_LOG - .info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIp(), dataId, group, tenant, requestIpAppName, + .info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIP(), dataId, group, tenant, requestIpAppName, ts, handleIp, "dump-all", type, -1); } @@ -196,7 +196,7 @@ public class ConfigTraceService { } //localIp | dataid | group | tenant| requestIpAppName| ts | event | type | [delayed] | ext(clientIp) LogUtil.TRACE_LOG - .info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIp(), dataId, group, tenant, requestIpAppName, + .info("{}|{}|{}|{}|{}|{}|{}|{}|{}|{}", InetUtils.getSelfIP(), dataId, group, tenant, requestIpAppName, ts, "pull", type, delayed, clientIp); } } diff --git a/config/src/main/java/com/alibaba/nacos/config/server/utils/DiskUtil.java b/config/src/main/java/com/alibaba/nacos/config/server/utils/DiskUtil.java index abfaae418..9096a84c6 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/utils/DiskUtil.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/utils/DiskUtil.java @@ -19,7 +19,7 @@ package com.alibaba.nacos.config.server.utils; import com.alibaba.nacos.common.utils.IoUtils; import com.alibaba.nacos.common.utils.MD5Utils; import com.alibaba.nacos.config.server.constant.Constants; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.io.FileUtils; import org.apache.commons.lang3.StringUtils; diff --git a/config/src/main/java/com/alibaba/nacos/config/server/utils/NamespaceUtil.java b/config/src/main/java/com/alibaba/nacos/config/server/utils/NamespaceUtil.java new file mode 100644 index 000000000..2fc0277ef --- /dev/null +++ b/config/src/main/java/com/alibaba/nacos/config/server/utils/NamespaceUtil.java @@ -0,0 +1,47 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.config.server.utils; + +import org.apache.commons.lang3.StringUtils; + +/** + * namespace(tenant) util. + * Because config and naming treat namespace(tenant) differently, + * this tool class can only be used by the config module. + * @author klw(213539@qq.com) + * @date 2020/10/12 17:56 + */ +public class NamespaceUtil { + + private static final String NAMESPACE_PUBLIC_KEY = "public"; + + private static final String NAMESPACE_NULL_KEY = "null"; + + /** + * Treat the namespace(tenant) parameters with values of "public" and "null" as an empty string. + * @param tenant namespace(tenant) id + * @return java.lang.String A namespace(tenant) string processed + */ + public static String processNamespaceParameter(String tenant) { + if (StringUtils.isBlank(tenant) || NAMESPACE_PUBLIC_KEY.equalsIgnoreCase(tenant) || NAMESPACE_NULL_KEY + .equalsIgnoreCase(tenant)) { + return ""; + } + return tenant.trim(); + } + +} diff --git a/config/src/main/java/com/alibaba/nacos/config/server/utils/PropertyUtil.java b/config/src/main/java/com/alibaba/nacos/config/server/utils/PropertyUtil.java index 91d7676c2..7a62a66b7 100644 --- a/config/src/main/java/com/alibaba/nacos/config/server/utils/PropertyUtil.java +++ b/config/src/main/java/com/alibaba/nacos/config/server/utils/PropertyUtil.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.config.server.utils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.slf4j.Logger; import org.springframework.context.ApplicationContextInitializer; import org.springframework.context.ConfigurableApplicationContext; diff --git a/config/src/test/java/com/alibaba/nacos/config/server/model/ConfigInfoTest.java b/config/src/test/java/com/alibaba/nacos/config/server/model/ConfigInfoTest.java index 321a69e9c..fb0225ed5 100644 --- a/config/src/test/java/com/alibaba/nacos/config/server/model/ConfigInfoTest.java +++ b/config/src/test/java/com/alibaba/nacos/config/server/model/ConfigInfoTest.java @@ -19,7 +19,7 @@ package com.alibaba.nacos.config.server.model; import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.consistency.IdGenerator; import com.alibaba.nacos.core.distributed.id.SnowFlowerIdGenerator; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Test; import org.springframework.core.env.StandardEnvironment; diff --git a/config/src/test/java/com/alibaba/nacos/config/server/service/ConfigChangePublisherTest.java b/config/src/test/java/com/alibaba/nacos/config/server/service/ConfigChangePublisherTest.java index 224e1e6fa..154106bb0 100644 --- a/config/src/test/java/com/alibaba/nacos/config/server/service/ConfigChangePublisherTest.java +++ b/config/src/test/java/com/alibaba/nacos/config/server/service/ConfigChangePublisherTest.java @@ -21,7 +21,7 @@ import com.alibaba.nacos.common.notify.NotifyCenter; import com.alibaba.nacos.common.notify.listener.Subscriber; import com.alibaba.nacos.config.server.model.event.ConfigDataChangeEvent; import com.alibaba.nacos.config.server.utils.PropertyUtil; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Test; diff --git a/config/src/test/java/com/alibaba/nacos/config/server/utils/NamespaceUtilTest.java b/config/src/test/java/com/alibaba/nacos/config/server/utils/NamespaceUtilTest.java new file mode 100644 index 000000000..16bbd700d --- /dev/null +++ b/config/src/test/java/com/alibaba/nacos/config/server/utils/NamespaceUtilTest.java @@ -0,0 +1,47 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.config.server.utils; + +import org.junit.Assert; +import org.junit.Test; + +/** + * test NamespaceUtil. + * + * @author klw(213539 @ qq.com) + * @date 2020/10/13 9:46 + */ +public class NamespaceUtilTest { + + @Test + public void testProcessTenantParameter() { + String strPublic = "public"; + String strNull = "null"; + String strEmpty = ""; + String strAbc = "abc"; + String strdef123 = "def123"; + String strAbcHasSpace = " abc "; + Assert.assertEquals(strEmpty, NamespaceUtil.processNamespaceParameter(strPublic)); + Assert.assertEquals(strEmpty, NamespaceUtil.processNamespaceParameter(strNull)); + Assert.assertEquals(strEmpty, NamespaceUtil.processNamespaceParameter(strEmpty)); + Assert.assertEquals(strEmpty, NamespaceUtil.processNamespaceParameter(null)); + Assert.assertEquals(strAbc, NamespaceUtil.processNamespaceParameter(strAbc)); + Assert.assertEquals(strdef123, NamespaceUtil.processNamespaceParameter(strdef123)); + Assert.assertEquals(strAbc, NamespaceUtil.processNamespaceParameter(strAbcHasSpace)); + } + +} diff --git a/consistency/src/main/java/com/alibaba/nacos/consistency/SerializeFactory.java b/consistency/src/main/java/com/alibaba/nacos/consistency/SerializeFactory.java index 32b8f94ac..eb95b2830 100644 --- a/consistency/src/main/java/com/alibaba/nacos/consistency/SerializeFactory.java +++ b/consistency/src/main/java/com/alibaba/nacos/consistency/SerializeFactory.java @@ -48,4 +48,7 @@ public class SerializeFactory { return SERIALIZER_MAP.get(defaultSerializer); } + public static Serializer getSerializer(String type) { + return SERIALIZER_MAP.get(type.toLowerCase()); + } } diff --git a/consistency/src/main/java/com/alibaba/nacos/consistency/Serializer.java b/consistency/src/main/java/com/alibaba/nacos/consistency/Serializer.java index aea9e01c0..41a69d318 100644 --- a/consistency/src/main/java/com/alibaba/nacos/consistency/Serializer.java +++ b/consistency/src/main/java/com/alibaba/nacos/consistency/Serializer.java @@ -16,6 +16,7 @@ package com.alibaba.nacos.consistency; +import java.lang.reflect.Type; import java.util.Map; import java.util.concurrent.ConcurrentHashMap; @@ -45,7 +46,17 @@ public interface Serializer { * @param class type * @return target object instance */ - T deserialize(byte[] data, Class cls); + T deserialize(byte[] data, Class cls); + + /** + * Deserialize the data. + * + * @param data byte[] + * @param type data type + * @param class type + * @return target object instance + */ + T deserialize(byte[] data, Type type); /** * Deserialize the data. diff --git a/consistency/src/main/java/com/alibaba/nacos/consistency/serialize/HessianSerializer.java b/consistency/src/main/java/com/alibaba/nacos/consistency/serialize/HessianSerializer.java index ca4c6d8eb..f25153a04 100644 --- a/consistency/src/main/java/com/alibaba/nacos/consistency/serialize/HessianSerializer.java +++ b/consistency/src/main/java/com/alibaba/nacos/consistency/serialize/HessianSerializer.java @@ -25,6 +25,7 @@ import com.caucho.hessian.io.SerializerFactory; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; import java.io.IOException; +import java.lang.reflect.Type; /** * hessian serializer. @@ -41,29 +42,24 @@ public class HessianSerializer implements Serializer { @Override public T deserialize(byte[] data) { - if (ByteUtils.isEmpty(data)) { - return null; - } - - Hessian2Input input = new Hessian2Input(new ByteArrayInputStream(data)); - input.setSerializerFactory(serializerFactory); - Object resultObject; - try { - resultObject = input.readObject(); - input.close(); - } catch (IOException e) { - throw new RuntimeException("IOException occurred when Hessian serializer decode!", e); - } - return (T) resultObject; + return deserialize(data); } @Override - public T deserialize(byte[] data, Class cls) { - + public T deserialize(byte[] data, Class cls) { + return deserialize(data); + } + + @Override + public T deserialize(byte[] data, Type type) { + return deserialize(data); + } + + private T deseiralize(byte[] data) { if (ByteUtils.isEmpty(data)) { return null; } - + Hessian2Input input = new Hessian2Input(new ByteArrayInputStream(data)); input.setSerializerFactory(serializerFactory); Object resultObject; @@ -96,4 +92,4 @@ public class HessianSerializer implements Serializer { return "Hessian"; } -} \ No newline at end of file +} diff --git a/consistency/src/main/java/com/alibaba/nacos/consistency/serialize/JacksonSerializer.java b/consistency/src/main/java/com/alibaba/nacos/consistency/serialize/JacksonSerializer.java new file mode 100644 index 000000000..17f6ff7f8 --- /dev/null +++ b/consistency/src/main/java/com/alibaba/nacos/consistency/serialize/JacksonSerializer.java @@ -0,0 +1,62 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.consistency.serialize; + +import com.alibaba.nacos.common.utils.ByteUtils; +import com.alibaba.nacos.common.utils.JacksonUtils; +import com.alibaba.nacos.consistency.Serializer; + +import java.lang.reflect.Type; + +/** + * Serializer implement by jackson. + * + * @author xiweng.yy + */ +public class JacksonSerializer implements Serializer { + + @Override + public T deserialize(byte[] data) { + throw new UnsupportedOperationException("Jackson serializer can't support deserialize json without type"); + } + + @Override + public T deserialize(byte[] data, Class cls) { + if (ByteUtils.isEmpty(data)) { + return null; + } + return JacksonUtils.toObj(data, cls); + } + + @Override + public T deserialize(byte[] data, Type type) { + if (ByteUtils.isEmpty(data)) { + return null; + } + return JacksonUtils.toObj(data, type); + } + + @Override + public byte[] serialize(T obj) { + return JacksonUtils.toJsonBytes(obj); + } + + @Override + public String name() { + return "JSON"; + } +} diff --git a/api/src/main/resources/application.properties b/consistency/src/main/resources/META-INF/services/com.alibaba.nacos.consistency.Serializer similarity index 91% rename from api/src/main/resources/application.properties rename to consistency/src/main/resources/META-INF/services/com.alibaba.nacos.consistency.Serializer index 0b3684628..26eec4d57 100644 --- a/api/src/main/resources/application.properties +++ b/consistency/src/main/resources/META-INF/services/com.alibaba.nacos.consistency.Serializer @@ -13,4 +13,5 @@ # See the License for the specific language governing permissions and # limitations under the License. # -version=${project.version} + +com.alibaba.nacos.consistency.serialize.JacksonSerializer diff --git a/console/src/main/java/com/alibaba/nacos/console/controller/ServerStateController.java b/console/src/main/java/com/alibaba/nacos/console/controller/ServerStateController.java index c1fc820c2..d889d1ad3 100644 --- a/console/src/main/java/com/alibaba/nacos/console/controller/ServerStateController.java +++ b/console/src/main/java/com/alibaba/nacos/console/controller/ServerStateController.java @@ -17,7 +17,7 @@ package com.alibaba.nacos.console.controller; import com.alibaba.nacos.common.utils.VersionUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RequestMapping; diff --git a/console/src/main/resources/application.properties b/console/src/main/resources/application.properties index 6ee15e18f..e06ad3bfe 100644 --- a/console/src/main/resources/application.properties +++ b/console/src/main/resources/application.properties @@ -175,3 +175,4 @@ nacos.istio.mcp.server.enabled=false # nacos.core.protocol.raft.data.read_index_type=ReadOnlySafe ### rpc request timeout, default 5 seconds # nacos.core.protocol.raft.data.rpc_request_timeout_ms=5000 + diff --git a/core/src/main/java/com/alibaba/nacos/core/auth/AuthFilter.java b/core/src/main/java/com/alibaba/nacos/core/auth/AuthFilter.java index 95fc13a19..535778bb3 100644 --- a/core/src/main/java/com/alibaba/nacos/core/auth/AuthFilter.java +++ b/core/src/main/java/com/alibaba/nacos/core/auth/AuthFilter.java @@ -24,7 +24,7 @@ import com.alibaba.nacos.auth.model.Permission; import com.alibaba.nacos.auth.parser.ResourceParser; import com.alibaba.nacos.common.utils.ExceptionUtil; import com.alibaba.nacos.core.code.ControllerMethodsCache; -import com.alibaba.nacos.core.utils.Constants; +import com.alibaba.nacos.sys.env.Constants; import com.alibaba.nacos.core.utils.Loggers; import com.alibaba.nacos.core.utils.WebUtils; import org.apache.commons.lang3.StringUtils; diff --git a/core/src/main/java/com/alibaba/nacos/core/auth/condition/PathRequestCondition.java b/core/src/main/java/com/alibaba/nacos/core/auth/condition/PathRequestCondition.java index 6af1ff269..1c674d2ae 100644 --- a/core/src/main/java/com/alibaba/nacos/core/auth/condition/PathRequestCondition.java +++ b/core/src/main/java/com/alibaba/nacos/core/auth/condition/PathRequestCondition.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.core.auth.condition; -import static com.alibaba.nacos.core.utils.Constants.REQUEST_PATH_SEPARATOR; +import static com.alibaba.nacos.sys.env.Constants.REQUEST_PATH_SEPARATOR; /** * request path info. method:{@link org.springframework.web.bind.annotation.RequestMapping#method()} path: {@link diff --git a/core/src/main/java/com/alibaba/nacos/core/cluster/Member.java b/core/src/main/java/com/alibaba/nacos/core/cluster/Member.java index aaed2c33d..3bea56abe 100644 --- a/core/src/main/java/com/alibaba/nacos/core/cluster/Member.java +++ b/core/src/main/java/com/alibaba/nacos/core/cluster/Member.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.core.cluster; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.lang3.StringUtils; import java.util.Collections; diff --git a/core/src/main/java/com/alibaba/nacos/core/cluster/MemberMetaDataConstants.java b/core/src/main/java/com/alibaba/nacos/core/cluster/MemberMetaDataConstants.java index 1d93db643..82343d1c3 100644 --- a/core/src/main/java/com/alibaba/nacos/core/cluster/MemberMetaDataConstants.java +++ b/core/src/main/java/com/alibaba/nacos/core/cluster/MemberMetaDataConstants.java @@ -38,9 +38,5 @@ public class MemberMetaDataConstants { public static final String VERSION = "version"; - public static final String[] META_KEY_LIST = new String[] {SITE_KEY, AD_WEIGHT, RAFT_PORT, WEIGHT, - LAST_REFRESH_TIME, VERSION}; - - public static final String[] META_KEY_LIST_WITHOUT_LAST_REFRESH_TIME = new String[] {SITE_KEY, AD_WEIGHT, RAFT_PORT, - WEIGHT, VERSION}; + public static final String[] BASIC_META_KEYS = new String[] {SITE_KEY, AD_WEIGHT, RAFT_PORT, WEIGHT, VERSION}; } diff --git a/core/src/main/java/com/alibaba/nacos/core/cluster/MemberUtils.java b/core/src/main/java/com/alibaba/nacos/core/cluster/MemberUtils.java index fa88a1603..f7eb4790c 100644 --- a/core/src/main/java/com/alibaba/nacos/core/cluster/MemberUtils.java +++ b/core/src/main/java/com/alibaba/nacos/core/cluster/MemberUtils.java @@ -17,8 +17,8 @@ package com.alibaba.nacos.core.cluster; import com.alibaba.nacos.common.utils.ExceptionUtil; -import com.alibaba.nacos.core.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.Loggers; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.lang3.StringUtils; import java.time.LocalDateTime; @@ -28,6 +28,7 @@ import java.util.HashMap; import java.util.HashSet; import java.util.List; import java.util.Map; +import java.util.Objects; import java.util.Set; import java.util.concurrent.ThreadLocalRandom; import java.util.function.Predicate; @@ -236,40 +237,40 @@ public class MemberUtils { } /** - * Judge whether two member is full equals. + * Judge whether basic info has changed. * * @param actual actual member * @param expected expected member * @return true if all content is same, otherwise false */ - public static boolean fullEquals(Member actual, Member expected) { + public static boolean isBasicInfoChanged(Member actual, Member expected) { if (null == expected) { return null == actual; } if (!expected.getIp().equals(actual.getIp())) { - return false; + return true; } if (expected.getPort() != actual.getPort()) { - return false; + return true; } if (!expected.getAddress().equals(actual.getAddress())) { - return false; + return true; } if (!expected.getState().equals(actual.getState())) { - return false; + return true; } - return equalsExtendInfo(expected, actual); + return isBasicInfoChangedInExtendInfo(expected, actual); } - private static boolean equalsExtendInfo(Member expected, Member actual) { - for (String each : MemberMetaDataConstants.META_KEY_LIST_WITHOUT_LAST_REFRESH_TIME) { + private static boolean isBasicInfoChangedInExtendInfo(Member expected, Member actual) { + for (String each : MemberMetaDataConstants.BASIC_META_KEYS) { if (expected.getExtendInfo().containsKey(each) != actual.getExtendInfo().containsKey(each)) { - return false; + return true; } - if (null != expected.getExtendVal(each) && !expected.getExtendVal(each).equals(actual.getExtendVal(each))) { - return false; + if (!Objects.equals(expected.getExtendVal(each), actual.getExtendVal(each))) { + return true; } } - return true; + return false; } } diff --git a/core/src/main/java/com/alibaba/nacos/core/cluster/ServerMemberManager.java b/core/src/main/java/com/alibaba/nacos/core/cluster/ServerMemberManager.java index 77ac3a83a..5af963296 100644 --- a/core/src/main/java/com/alibaba/nacos/core/cluster/ServerMemberManager.java +++ b/core/src/main/java/com/alibaba/nacos/core/cluster/ServerMemberManager.java @@ -32,13 +32,13 @@ import com.alibaba.nacos.common.utils.ConcurrentHashSet; import com.alibaba.nacos.common.utils.ExceptionUtil; import com.alibaba.nacos.common.utils.VersionUtils; import com.alibaba.nacos.core.cluster.lookup.LookupFactory; -import com.alibaba.nacos.core.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.Commons; -import com.alibaba.nacos.core.utils.Constants; import com.alibaba.nacos.core.utils.GenericType; import com.alibaba.nacos.core.utils.GlobalExecutor; -import com.alibaba.nacos.core.utils.InetUtils; import com.alibaba.nacos.core.utils.Loggers; +import com.alibaba.nacos.sys.env.Constants; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.apache.commons.lang3.StringUtils; import org.springframework.boot.web.context.WebServerInitializedEvent; import org.springframework.context.ApplicationListener; @@ -78,7 +78,8 @@ import java.util.concurrent.ConcurrentSkipListMap; @Component(value = "serverMemberManager") public class ServerMemberManager implements ApplicationListener { - private final NacosAsyncRestTemplate asyncRestTemplate = HttpClientBeanHolder.getNacosAsyncRestTemplate(Loggers.CORE); + private final NacosAsyncRestTemplate asyncRestTemplate = HttpClientBeanHolder + .getNacosAsyncRestTemplate(Loggers.CORE); /** * Cluster node list. @@ -131,7 +132,7 @@ public class ServerMemberManager implements ApplicationListener() { @Override public void onEvent(InetUtils.IPChangeEvent event) { - String newAddress = event.getNewIp() + ":" + port; + String newAddress = event.getNewIP() + ":" + port; ServerMemberManager.this.localAddress = newAddress; ApplicationUtils.setLocalAddress(localAddress); - + Member self = ServerMemberManager.this.self; - self.setIp(event.getNewIp()); - - String oldAddress = event.getOldIp() + ":" + port; + self.setIp(event.getNewIP()); + + String oldAddress = event.getOldIP() + ":" + port; ServerMemberManager.this.serverList.remove(oldAddress); ServerMemberManager.this.serverList.put(newAddress, self); @@ -209,10 +210,11 @@ public class ServerMemberManager implements ApplicationListener= maxFailCount) { diff --git a/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/FileConfigMemberLookup.java b/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/FileConfigMemberLookup.java index f51294db0..4228119a7 100644 --- a/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/FileConfigMemberLookup.java +++ b/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/FileConfigMemberLookup.java @@ -20,11 +20,11 @@ import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.core.cluster.AbstractMemberLookup; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.MemberUtils; -import com.alibaba.nacos.core.file.FileChangeEvent; -import com.alibaba.nacos.core.file.FileWatcher; -import com.alibaba.nacos.core.file.WatchFileCenter; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.file.FileChangeEvent; +import com.alibaba.nacos.sys.file.FileWatcher; +import com.alibaba.nacos.sys.file.WatchFileCenter; import com.alibaba.nacos.core.utils.Loggers; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.lang3.StringUtils; import java.util.ArrayList; diff --git a/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/LookupFactory.java b/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/LookupFactory.java index 710ee598b..68fd26938 100644 --- a/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/LookupFactory.java +++ b/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/LookupFactory.java @@ -20,7 +20,7 @@ import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.common.utils.StringUtils; import com.alibaba.nacos.core.cluster.MemberLookup; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.Loggers; import java.io.File; diff --git a/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/StandaloneMemberLookup.java b/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/StandaloneMemberLookup.java index 5bc3fccef..47dabbc5d 100644 --- a/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/StandaloneMemberLookup.java +++ b/core/src/main/java/com/alibaba/nacos/core/cluster/lookup/StandaloneMemberLookup.java @@ -17,6 +17,9 @@ package com.alibaba.nacos.core.cluster.lookup; import com.alibaba.nacos.core.cluster.AbstractMemberLookup; +import com.alibaba.nacos.core.cluster.MemberUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import java.util.Collections; @@ -30,7 +33,8 @@ public class StandaloneMemberLookup extends AbstractMemberLookup { @Override public void start() { if (start.compareAndSet(false, true)) { - afterLookup(Collections.singletonList(memberManager.getSelf())); + String url = InetUtils.getSelfIP() + ":" + ApplicationUtils.getPort(); + afterLookup(MemberUtils.readServerConf(Collections.singletonList(url))); } } } diff --git a/core/src/main/java/com/alibaba/nacos/core/code/ControllerMethodsCache.java b/core/src/main/java/com/alibaba/nacos/core/code/ControllerMethodsCache.java index 690ba1de4..9ddef6117 100644 --- a/core/src/main/java/com/alibaba/nacos/core/code/ControllerMethodsCache.java +++ b/core/src/main/java/com/alibaba/nacos/core/code/ControllerMethodsCache.java @@ -45,7 +45,8 @@ import java.util.Set; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap; -import static com.alibaba.nacos.core.utils.Constants.REQUEST_PATH_SEPARATOR; +import static com.alibaba.nacos.sys.env.Constants.REQUEST_PATH_SEPARATOR; + /** * Method cache. diff --git a/core/src/main/java/com/alibaba/nacos/core/code/LoggingSpringApplicationRunListener.java b/core/src/main/java/com/alibaba/nacos/core/code/LoggingSpringApplicationRunListener.java index 37b052f87..780d8e6fb 100644 --- a/core/src/main/java/com/alibaba/nacos/core/code/LoggingSpringApplicationRunListener.java +++ b/core/src/main/java/com/alibaba/nacos/core/code/LoggingSpringApplicationRunListener.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.core.code; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.boot.SpringApplication; diff --git a/core/src/main/java/com/alibaba/nacos/core/code/StandaloneProfileApplicationListener.java b/core/src/main/java/com/alibaba/nacos/core/code/StandaloneProfileApplicationListener.java index 10d733894..e741db0e4 100644 --- a/core/src/main/java/com/alibaba/nacos/core/code/StandaloneProfileApplicationListener.java +++ b/core/src/main/java/com/alibaba/nacos/core/code/StandaloneProfileApplicationListener.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.core.code; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.boot.context.event.ApplicationEnvironmentPreparedEvent; @@ -27,8 +27,8 @@ import org.springframework.core.env.ConfigurableEnvironment; import java.util.Arrays; -import static com.alibaba.nacos.core.utils.Constants.STANDALONE_MODE_PROPERTY_NAME; -import static com.alibaba.nacos.core.utils.Constants.STANDALONE_SPRING_PROFILE; +import static com.alibaba.nacos.sys.env.Constants.STANDALONE_MODE_PROPERTY_NAME; +import static com.alibaba.nacos.sys.env.Constants.STANDALONE_SPRING_PROFILE; /** * Standalone {@link Profile} {@link ApplicationListener} for {@link ApplicationEnvironmentPreparedEvent}. diff --git a/core/src/main/java/com/alibaba/nacos/core/code/StartingSpringApplicationRunListener.java b/core/src/main/java/com/alibaba/nacos/core/code/StartingSpringApplicationRunListener.java index 0b8c69777..12804721b 100644 --- a/core/src/main/java/com/alibaba/nacos/core/code/StartingSpringApplicationRunListener.java +++ b/core/src/main/java/com/alibaba/nacos/core/code/StartingSpringApplicationRunListener.java @@ -21,10 +21,10 @@ import com.alibaba.nacos.common.executor.NameThreadFactory; import com.alibaba.nacos.common.executor.ThreadPoolManager; import com.alibaba.nacos.common.http.HttpClientManager; import com.alibaba.nacos.common.notify.NotifyCenter; -import com.alibaba.nacos.core.file.WatchFileCenter; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.DiskUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.file.WatchFileCenter; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.boot.SpringApplication; @@ -86,7 +86,7 @@ public class StartingSpringApplicationRunListener implements SpringApplicationRu System.setProperty(MODE_PROPERTY_KEY_FUNCTION_MODE, ApplicationUtils.FUNCTION_MODE_NAMING); } - System.setProperty(LOCAL_IP_PROPERTY_KEY, InetUtils.getSelfIp()); + System.setProperty(LOCAL_IP_PROPERTY_KEY, InetUtils.getSelfIP()); } @Override diff --git a/core/src/main/java/com/alibaba/nacos/core/controller/NacosClusterController.java b/core/src/main/java/com/alibaba/nacos/core/controller/NacosClusterController.java index fc54d67a6..50f3a02e5 100644 --- a/core/src/main/java/com/alibaba/nacos/core/controller/NacosClusterController.java +++ b/core/src/main/java/com/alibaba/nacos/core/controller/NacosClusterController.java @@ -29,7 +29,7 @@ import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.MemberUtils; import com.alibaba.nacos.core.cluster.NodeState; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.Commons; import com.alibaba.nacos.core.utils.GenericType; import com.alibaba.nacos.core.utils.Loggers; diff --git a/core/src/main/java/com/alibaba/nacos/core/distributed/ProtocolManager.java b/core/src/main/java/com/alibaba/nacos/core/distributed/ProtocolManager.java index bf2729027..0c434cf44 100644 --- a/core/src/main/java/com/alibaba/nacos/core/distributed/ProtocolManager.java +++ b/core/src/main/java/com/alibaba/nacos/core/distributed/ProtocolManager.java @@ -26,7 +26,7 @@ import com.alibaba.nacos.core.cluster.MemberChangeListener; import com.alibaba.nacos.core.cluster.MemberMetaDataConstants; import com.alibaba.nacos.core.cluster.MemberUtils; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.ClassUtils; import org.springframework.beans.factory.DisposableBean; import org.springframework.beans.factory.annotation.Autowired; diff --git a/core/src/main/java/com/alibaba/nacos/core/distributed/distro/task/load/DistroLoadDataTask.java b/core/src/main/java/com/alibaba/nacos/core/distributed/distro/task/load/DistroLoadDataTask.java index b919b6d70..35f0ec7ae 100644 --- a/core/src/main/java/com/alibaba/nacos/core/distributed/distro/task/load/DistroLoadDataTask.java +++ b/core/src/main/java/com/alibaba/nacos/core/distributed/distro/task/load/DistroLoadDataTask.java @@ -109,7 +109,7 @@ public class DistroLoadDataTask implements Runnable { return true; } } catch (Exception e) { - Loggers.DISTRO.error("[DISTRO-INIT] load snapshot {} from {} failed.", resourceType, each.getAddress()); + Loggers.DISTRO.error("[DISTRO-INIT] load snapshot {} from {} failed.", resourceType, each.getAddress(), e); } } return false; diff --git a/core/src/main/java/com/alibaba/nacos/core/distributed/id/SnowFlowerIdGenerator.java b/core/src/main/java/com/alibaba/nacos/core/distributed/id/SnowFlowerIdGenerator.java index 7f483e987..b8c38e1e3 100644 --- a/core/src/main/java/com/alibaba/nacos/core/distributed/id/SnowFlowerIdGenerator.java +++ b/core/src/main/java/com/alibaba/nacos/core/distributed/id/SnowFlowerIdGenerator.java @@ -17,7 +17,7 @@ package com.alibaba.nacos.core.distributed.id; import com.alibaba.nacos.consistency.IdGenerator; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.google.common.base.Preconditions; import org.slf4j.Logger; import org.slf4j.LoggerFactory; diff --git a/core/src/main/java/com/alibaba/nacos/core/distributed/raft/JRaftServer.java b/core/src/main/java/com/alibaba/nacos/core/distributed/raft/JRaftServer.java index e3073e421..9938e2f3b 100644 --- a/core/src/main/java/com/alibaba/nacos/core/distributed/raft/JRaftServer.java +++ b/core/src/main/java/com/alibaba/nacos/core/distributed/raft/JRaftServer.java @@ -39,7 +39,7 @@ import com.alibaba.nacos.core.distributed.raft.utils.JRaftUtils; import com.alibaba.nacos.core.distributed.raft.utils.RaftExecutor; import com.alibaba.nacos.core.distributed.raft.utils.RaftOptionsBuilder; import com.alibaba.nacos.core.monitor.MetricsMonitor; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.Loggers; import com.alipay.sofa.jraft.CliService; import com.alipay.sofa.jraft.Node; @@ -497,17 +497,17 @@ public class JRaftServer { Configuration oldConf = instance.getConfiguration(groupName); String oldLeader = Optional.ofNullable(instance.selectLeader(groupName)).orElse(PeerId.emptyPeer()) .getEndpoint().toString(); - status = instance.refreshConfiguration(this.cliClientService, groupName, rpcRequestTimeoutMs); - if (!status.isOk()) { - Loggers.RAFT - .error("Fail to refresh route configuration for group : {}, status is : {}", groupName, status); - } // fix issue #3661 https://github.com/alibaba/nacos/issues/3661 status = instance.refreshLeader(this.cliClientService, groupName, rpcRequestTimeoutMs); if (!status.isOk()) { Loggers.RAFT .error("Fail to refresh leader for group : {}, status is : {}", groupName, status); } + status = instance.refreshConfiguration(this.cliClientService, groupName, rpcRequestTimeoutMs); + if (!status.isOk()) { + Loggers.RAFT + .error("Fail to refresh route configuration for group : {}, status is : {}", groupName, status); + } } catch (Exception e) { Loggers.RAFT.error("Fail to refresh raft metadata info for group : {}, error is : {}", groupName, e); } diff --git a/core/src/main/java/com/alibaba/nacos/core/distributed/raft/utils/JRaftUtils.java b/core/src/main/java/com/alibaba/nacos/core/distributed/raft/utils/JRaftUtils.java index c672f6b36..02fb26bbf 100644 --- a/core/src/main/java/com/alibaba/nacos/core/distributed/raft/utils/JRaftUtils.java +++ b/core/src/main/java/com/alibaba/nacos/core/distributed/raft/utils/JRaftUtils.java @@ -25,9 +25,9 @@ import com.alibaba.nacos.core.cluster.ServerMemberManager; import com.alibaba.nacos.core.distributed.raft.JRaftServer; import com.alibaba.nacos.core.distributed.raft.processor.NacosGetRequestProcessor; import com.alibaba.nacos.core.distributed.raft.processor.NacosLogProcessor; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.Loggers; +import com.alibaba.nacos.sys.utils.DiskUtils; import com.alipay.sofa.jraft.CliService; import com.alipay.sofa.jraft.RouteTable; import com.alipay.sofa.jraft.Status; diff --git a/core/src/main/java/com/alibaba/nacos/core/storage/kv/FileKvStorage.java b/core/src/main/java/com/alibaba/nacos/core/storage/kv/FileKvStorage.java index 4257ab6da..b730a219d 100644 --- a/core/src/main/java/com/alibaba/nacos/core/storage/kv/FileKvStorage.java +++ b/core/src/main/java/com/alibaba/nacos/core/storage/kv/FileKvStorage.java @@ -16,14 +16,16 @@ package com.alibaba.nacos.core.storage.kv; +import com.alibaba.nacos.common.utils.ByteUtils; import com.alibaba.nacos.core.exception.ErrorCode; import com.alibaba.nacos.core.exception.KvStorageException; -import com.alibaba.nacos.core.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; import java.io.File; import java.io.IOException; import java.nio.file.Paths; import java.util.HashMap; +import java.util.LinkedList; import java.util.List; import java.util.Map; import java.util.concurrent.locks.ReentrantReadWriteLock; @@ -158,11 +160,14 @@ public class FileKvStorage implements KvStorage { public void snapshotLoad(String path) throws KvStorageException { writeLock.lock(); try { - // First clean up the local file information, before the file copy - DiskUtils.deleteDirThenMkdir(baseDir); File srcDir = Paths.get(path).toFile(); - File descDir = Paths.get(baseDir).toFile(); - DiskUtils.copyDirectory(srcDir, descDir); + // If snapshot path is non-exist, means snapshot is empty + if (srcDir.exists()) { + // First clean up the local file information, before the file copy + DiskUtils.deleteDirThenMkdir(baseDir); + File descDir = Paths.get(baseDir).toFile(); + DiskUtils.copyDirectory(srcDir, descDir); + } } catch (IOException e) { throw new KvStorageException(ErrorCode.IOCopyDirError, e); } finally { @@ -171,7 +176,20 @@ public class FileKvStorage implements KvStorage { } @Override - public void shutdown() { - + public List allKeys() throws KvStorageException { + List result = new LinkedList<>(); + File[] files = new File(baseDir).listFiles(); + if (null != files) { + for (File each : files) { + if (each.isFile()) { + result.add(ByteUtils.toBytes(each.getName())); + } + } + } + return result; } -} + + @Override + public void shutdown() { + } +} \ No newline at end of file diff --git a/core/src/main/java/com/alibaba/nacos/core/storage/kv/KvStorage.java b/core/src/main/java/com/alibaba/nacos/core/storage/kv/KvStorage.java index 7e1d4df82..38ddd5cfb 100644 --- a/core/src/main/java/com/alibaba/nacos/core/storage/kv/KvStorage.java +++ b/core/src/main/java/com/alibaba/nacos/core/storage/kv/KvStorage.java @@ -114,6 +114,14 @@ public interface KvStorage { */ void snapshotLoad(String path) throws KvStorageException; + /** + * Get all keys. + * + * @return all keys + * @throws KvStorageException KVStorageException + */ + List allKeys() throws KvStorageException; + /** * shutdown. */ diff --git a/core/src/main/java/com/alibaba/nacos/core/storage/kv/MemoryKvStorage.java b/core/src/main/java/com/alibaba/nacos/core/storage/kv/MemoryKvStorage.java index 2b77868da..34577f701 100644 --- a/core/src/main/java/com/alibaba/nacos/core/storage/kv/MemoryKvStorage.java +++ b/core/src/main/java/com/alibaba/nacos/core/storage/kv/MemoryKvStorage.java @@ -22,6 +22,7 @@ import com.alipay.sofa.jraft.util.BytesUtil; import java.util.Arrays; import java.util.HashMap; +import java.util.LinkedList; import java.util.List; import java.util.Map; import java.util.concurrent.ConcurrentSkipListMap; @@ -91,12 +92,21 @@ public class MemoryKvStorage implements KvStorage { throw new UnsupportedOperationException(); } + @Override + public List allKeys() throws KvStorageException { + List result = new LinkedList<>(); + for (Key each : storage.keySet()) { + result.add(each.origin); + } + return result; + } + @Override public void shutdown() { storage.clear(); } - private static class Key implements Comparable { + private static class Key implements Comparable { private final byte[] origin; @@ -105,8 +115,8 @@ public class MemoryKvStorage implements KvStorage { } @Override - public int compareTo(byte[] o) { - return BytesUtil.compare(origin, o); + public int compareTo(Key o) { + return BytesUtil.compare(origin, o.origin); } @Override diff --git a/core/src/main/java/com/alibaba/nacos/core/utils/PropertyUtil.java b/core/src/main/java/com/alibaba/nacos/core/utils/PropertyUtil.java deleted file mode 100644 index cb95202e0..000000000 --- a/core/src/main/java/com/alibaba/nacos/core/utils/PropertyUtil.java +++ /dev/null @@ -1,81 +0,0 @@ -/* - * Copyright 1999-2018 Alibaba Group Holding Ltd. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package com.alibaba.nacos.core.utils; - -import org.apache.commons.io.IOUtils; -import org.apache.commons.lang3.StringUtils; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.FileInputStream; -import java.io.InputStream; -import java.util.ArrayList; -import java.util.List; -import java.util.Properties; - -/** - * operation property. - * - * @author Nacos - */ -public class PropertyUtil { - - private static final Logger LOGGER = LoggerFactory.getLogger(PropertyUtil.class); - - private static Properties properties = new Properties(); - - static { - InputStream inputStream = null; - try { - String baseDir = System.getProperty("nacos.home"); - if (StringUtils.isNotBlank(baseDir)) { - inputStream = new FileInputStream(baseDir + "/conf/application.properties"); - } else { - inputStream = PropertyUtil.class.getResourceAsStream("/application.properties"); - } - properties.load(inputStream); - } catch (Exception e) { - LOGGER.error("read property file error:" + e); - } finally { - IOUtils.closeQuietly(inputStream); - } - } - - public static String getProperty(String key) { - return properties.getProperty(key); - } - - public static String getProperty(String key, String defaultValue) { - return properties.getProperty(key, defaultValue); - } - - public static List getPropertyList(String key) { - List valueList = new ArrayList<>(); - - for (int i = 0; i < Integer.MAX_VALUE; i++) { - String value = properties.getProperty(key + "[" + i + "]"); - if (StringUtils.isBlank(value)) { - break; - } - - valueList.add(value); - } - - return valueList; - } - -} diff --git a/core/src/main/java/com/alibaba/nacos/core/utils/TimerContext.java b/core/src/main/java/com/alibaba/nacos/core/utils/TimerContext.java index 157e81c26..6b76ab916 100644 --- a/core/src/main/java/com/alibaba/nacos/core/utils/TimerContext.java +++ b/core/src/main/java/com/alibaba/nacos/core/utils/TimerContext.java @@ -16,11 +16,13 @@ package com.alibaba.nacos.core.utils; +import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.common.utils.LoggerUtils; import com.alibaba.nacos.common.utils.Pair; import com.alibaba.nacos.common.utils.StringUtils; import org.slf4j.Logger; +import java.util.concurrent.Callable; import java.util.function.Consumer; import java.util.function.Function; import java.util.function.Supplier; diff --git a/core/src/main/java/com/alibaba/nacos/core/utils/WebUtils.java b/core/src/main/java/com/alibaba/nacos/core/utils/WebUtils.java index ae2874e5e..e8d89917c 100644 --- a/core/src/main/java/com/alibaba/nacos/core/utils/WebUtils.java +++ b/core/src/main/java/com/alibaba/nacos/core/utils/WebUtils.java @@ -20,6 +20,7 @@ import com.alibaba.nacos.common.constant.HttpHeaderConsts; import com.alibaba.nacos.common.http.HttpUtils; import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.model.RestResultUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; import org.apache.commons.lang3.StringUtils; import org.springframework.web.context.request.async.DeferredResult; import org.springframework.web.multipart.MultipartFile; diff --git a/core/src/main/resources/META-INF/logback/nacos.xml b/core/src/main/resources/META-INF/logback/nacos.xml index 9bbbf7aec..68b327681 100644 --- a/core/src/main/resources/META-INF/logback/nacos.xml +++ b/core/src/main/resources/META-INF/logback/nacos.xml @@ -205,7 +205,7 @@ - + diff --git a/core/src/main/resources/META-INF/spring.factories b/core/src/main/resources/META-INF/spring.factories index eb5377634..c63c723f2 100644 --- a/core/src/main/resources/META-INF/spring.factories +++ b/core/src/main/resources/META-INF/spring.factories @@ -1,15 +1,6 @@ -# ApplicationContextInitializer -org.springframework.context.ApplicationContextInitializer=\ - com.alibaba.nacos.core.utils.ApplicationUtils - # ApplicationListener org.springframework.context.ApplicationListener=\ com.alibaba.nacos.core.code.StandaloneProfileApplicationListener - -# EnvironmentPostProcessor -org.springframework.boot.env.EnvironmentPostProcessor=\ -com.alibaba.nacos.core.env.NacosDefaultPropertySourceEnvironmentPostProcessor - # SpringApplicationRunListener org.springframework.boot.SpringApplicationRunListener=\ com.alibaba.nacos.core.code.LoggingSpringApplicationRunListener,\ diff --git a/core/src/test/java/com/alibaba/nacos/core/cluster/MemberUtilsTest.java b/core/src/test/java/com/alibaba/nacos/core/cluster/MemberUtilsTest.java new file mode 100644 index 000000000..490d7153f --- /dev/null +++ b/core/src/test/java/com/alibaba/nacos/core/cluster/MemberUtilsTest.java @@ -0,0 +1,106 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.core.cluster; + +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.mockito.junit.MockitoJUnitRunner; +import org.springframework.core.env.ConfigurableEnvironment; + +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +@RunWith(MockitoJUnitRunner.class) +public class MemberUtilsTest { + + private static final String IP = "1.1.1.1"; + + private static final int PORT = 8848; + + @Mock + private ConfigurableEnvironment environment; + + private Member originalMember; + + @Before + public void setUp() { + ApplicationUtils.injectEnvironment(environment); + originalMember = buildMember(); + } + + private Member buildMember() { + return Member.builder().ip(IP).port(PORT).state(NodeState.UP).build(); + } + + @Test + public void testIsBasicInfoChangedNoChangeWithoutExtendInfo() { + Member newMember = buildMember(); + assertFalse(MemberUtils.isBasicInfoChanged(newMember, originalMember)); + } + + @Test + public void testIsBasicInfoChangedNoChangeWithExtendInfo() { + Member newMember = buildMember(); + newMember.setExtendVal("test", "test"); + assertFalse(MemberUtils.isBasicInfoChanged(newMember, originalMember)); + } + + @Test + public void testIsBasicInfoChangedForIp() { + Member newMember = buildMember(); + newMember.setIp("1.1.1.2"); + assertTrue(MemberUtils.isBasicInfoChanged(newMember, originalMember)); + } + + @Test + public void testIsBasicInfoChangedForPort() { + Member newMember = buildMember(); + newMember.setPort(PORT + 1); + assertTrue(MemberUtils.isBasicInfoChanged(newMember, originalMember)); + } + + @Test + public void testIsBasicInfoChangedForAddress() { + Member newMember = buildMember(); + newMember.setAddress("test"); + assertTrue(MemberUtils.isBasicInfoChanged(newMember, originalMember)); + } + + @Test + public void testIsBasicInfoChangedForStatus() { + Member newMember = buildMember(); + newMember.setState(NodeState.DOWN); + assertTrue(MemberUtils.isBasicInfoChanged(newMember, originalMember)); + } + + @Test + public void testIsBasicInfoChangedForMoreBasicExtendInfo() { + Member newMember = buildMember(); + newMember.setExtendVal(MemberMetaDataConstants.VERSION, "TEST"); + assertTrue(MemberUtils.isBasicInfoChanged(newMember, originalMember)); + } + + @Test + public void testIsBasicInfoChangedForChangedBasicExtendInfo() { + Member newMember = buildMember(); + newMember.setExtendVal(MemberMetaDataConstants.WEIGHT, "100"); + assertTrue(MemberUtils.isBasicInfoChanged(newMember, originalMember)); + } +} diff --git a/core/src/test/java/com/alibaba/nacos/core/cluster/ServerMemberManagerTest.java b/core/src/test/java/com/alibaba/nacos/core/cluster/ServerMemberManagerTest.java new file mode 100644 index 000000000..9d395b8f4 --- /dev/null +++ b/core/src/test/java/com/alibaba/nacos/core/cluster/ServerMemberManagerTest.java @@ -0,0 +1,111 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.core.cluster; + +import com.alibaba.nacos.api.exception.NacosException; +import com.alibaba.nacos.common.notify.EventPublisher; +import com.alibaba.nacos.common.notify.NotifyCenter; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.mockito.junit.MockitoJUnitRunner; +import org.springframework.core.env.ConfigurableEnvironment; + +import javax.servlet.ServletContext; +import java.util.concurrent.atomic.AtomicBoolean; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@RunWith(MockitoJUnitRunner.class) +public class ServerMemberManagerTest { + + @Mock + private ConfigurableEnvironment environment; + + @Mock + private ServletContext servletContext; + + @Mock + private EventPublisher eventPublisher; + + private ServerMemberManager serverMemberManager; + + private static final AtomicBoolean EVENT_PUBLISH = new AtomicBoolean(false); + + @Before + public void setUp() throws Exception { + when(environment.getProperty("server.port", Integer.class, 8848)).thenReturn(8848); + when(environment.getProperty("nacos.member-change-event.queue.size", Integer.class, 128)).thenReturn(128); + ApplicationUtils.injectEnvironment(environment); + when(servletContext.getContextPath()).thenReturn(""); + serverMemberManager = new ServerMemberManager(servletContext); + serverMemberManager.updateMember(Member.builder().ip("1.1.1.1").port(8848).state(NodeState.UP).build()); + serverMemberManager.getMemberAddressInfos().add("1.1.1.1:8848"); + NotifyCenter.getPublisherMap().put(MembersChangeEvent.class.getCanonicalName(), eventPublisher); + } + + @After + public void tearDown() throws NacosException { + EVENT_PUBLISH.set(false); + NotifyCenter.deregisterPublisher(MembersChangeEvent.class); + serverMemberManager.shutdown(); + } + + @Test + public void testUpdateNonExistMember() { + Member newMember = Member.builder().ip("1.1.1.2").port(8848).state(NodeState.UP).build(); + assertFalse(serverMemberManager.update(newMember)); + } + + @Test + public void testUpdateDownMember() { + Member newMember = Member.builder().ip("1.1.1.1").port(8848).state(NodeState.DOWN).build(); + assertTrue(serverMemberManager.update(newMember)); + assertFalse(serverMemberManager.getMemberAddressInfos().contains("1.1.1.1:8848")); + verify(eventPublisher).publish(any(MembersChangeEvent.class)); + } + + @Test + public void testUpdateVersionMember() { + Member newMember = Member.builder().ip("1.1.1.1").port(8848).state(NodeState.UP).build(); + newMember.setExtendVal(MemberMetaDataConstants.VERSION, "testVersion"); + assertTrue(serverMemberManager.update(newMember)); + assertTrue(serverMemberManager.getMemberAddressInfos().contains("1.1.1.1:8848")); + assertEquals("testVersion", + serverMemberManager.getServerList().get("1.1.1.1:8848").getExtendVal(MemberMetaDataConstants.VERSION)); + verify(eventPublisher).publish(any(MembersChangeEvent.class)); + } + + @Test + public void testUpdateNonBasicExtendInfoMember() { + Member newMember = Member.builder().ip("1.1.1.1").port(8848).state(NodeState.UP).build(); + newMember.setExtendVal("naming", "test"); + assertTrue(serverMemberManager.update(newMember)); + assertTrue(serverMemberManager.getMemberAddressInfos().contains("1.1.1.1:8848")); + assertEquals("test", serverMemberManager.getServerList().get("1.1.1.1:8848").getExtendVal("naming")); + verify(eventPublisher, never()).publish(any(MembersChangeEvent.class)); + } +} diff --git a/core/src/test/java/com/alibaba/nacos/core/listener/StandaloneProfileApplicationListenerTest.java b/core/src/test/java/com/alibaba/nacos/core/listener/StandaloneProfileApplicationListenerTest.java index d82703dcc..ebf12f347 100644 --- a/core/src/test/java/com/alibaba/nacos/core/listener/StandaloneProfileApplicationListenerTest.java +++ b/core/src/test/java/com/alibaba/nacos/core/listener/StandaloneProfileApplicationListenerTest.java @@ -27,7 +27,7 @@ import org.springframework.boot.test.context.SpringBootTest; import org.springframework.core.env.Environment; import org.springframework.test.context.junit4.SpringRunner; -import static com.alibaba.nacos.core.utils.Constants.STANDALONE_SPRING_PROFILE; +import static com.alibaba.nacos.sys.env.Constants.STANDALONE_SPRING_PROFILE; /** * {@link StandaloneProfileApplicationListener} Test. diff --git a/core/src/test/java/com/alibaba/nacos/core/storage/RocksStorageTest.java b/core/src/test/java/com/alibaba/nacos/core/storage/RocksStorageTest.java deleted file mode 100644 index 35c3cc4d1..000000000 --- a/core/src/test/java/com/alibaba/nacos/core/storage/RocksStorageTest.java +++ /dev/null @@ -1,76 +0,0 @@ -/* - * Copyright 1999-2018 Alibaba Group Holding Ltd. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ - -package com.alibaba.nacos.core.storage; - -import com.alibaba.nacos.common.utils.ThreadUtils; -import com.alibaba.nacos.core.storage.kv.RocksStorage; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.DiskUtils; -import org.junit.Assert; -import org.junit.Before; -import org.junit.Test; -import org.rocksdb.RocksDB; -import org.springframework.mock.env.MockEnvironment; - -import java.nio.file.Paths; - -public class RocksStorageTest { - - static { - ApplicationUtils.injectEnvironment(new MockEnvironment()); - RocksDB.loadLibrary(); - } - - static final String DIR = Paths.get(ApplicationUtils.getNacosTmpDir(), "rocksdb").toString(); - - @Before - public void before() throws Throwable { - DiskUtils.deleteDirectory(DIR); - } - - @Test - public void testCreateRocksStorage() throws Throwable { - RocksStorage storage = RocksStorage.createDefault("test", DIR); - storage.put("liaochuntao".getBytes(), "liaochuntao".getBytes()); - } - - @Test - public void testRocksStorageSnapshotSave() throws Throwable { - try { - RocksStorage storage = RocksStorage.createDefault("test", DIR); - storage.put("liaochuntao".getBytes(), "liaochuntao".getBytes()); - storage.doSnapshot(Paths.get(DIR, "snapshot").toString()); - } catch (Throwable ex) { - Assert.fail(ex.getMessage()); - } - } - - @Test - public void testRocksStorageSnapshotLoad() throws Throwable { - RocksStorage storage = RocksStorage.createDefault("test", DIR); - storage.put("liaochuntao".getBytes(), "liaochuntao".getBytes()); - storage.snapshotLoad(Paths.get(DIR, "snapshot").toString()); - storage.shutdown(); - ThreadUtils.sleep(5_000L); - storage = RocksStorage.createDefault("test", Paths.get(ApplicationUtils.getNacosTmpDir(), "snapshot_load").toString()); - storage.snapshotLoad(Paths.get(DIR, "snapshot").toString()); - byte[] b = storage.get("liaochuntao".getBytes()); - Assert.assertArrayEquals(b, "liaochuntao".getBytes()); - } - -} \ No newline at end of file diff --git a/core/src/test/java/com/alibaba/nacos/core/utils/SystemUtilsTest.java b/core/src/test/java/com/alibaba/nacos/core/utils/SystemUtilsTest.java index a60c826c2..5f6843d7a 100644 --- a/core/src/test/java/com/alibaba/nacos/core/utils/SystemUtilsTest.java +++ b/core/src/test/java/com/alibaba/nacos/core/utils/SystemUtilsTest.java @@ -16,6 +16,7 @@ package com.alibaba.nacos.core.utils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.io.FileUtils; import org.junit.Assert; import org.junit.BeforeClass; @@ -26,8 +27,8 @@ import java.io.IOException; import java.util.List; import java.util.Random; -import static com.alibaba.nacos.core.utils.Constants.PREFER_HOSTNAME_OVER_IP_PROPERTY_NAME; -import static com.alibaba.nacos.core.utils.Constants.STANDALONE_MODE_PROPERTY_NAME; +import static com.alibaba.nacos.sys.env.Constants.PREFER_HOSTNAME_OVER_IP_PROPERTY_NAME; +import static com.alibaba.nacos.sys.env.Constants.STANDALONE_MODE_PROPERTY_NAME; /** * {@link ApplicationUtils} Test. diff --git a/distribution/conf/nacos-logback.xml b/distribution/conf/nacos-logback.xml index 3779fbc66..b4a731117 100644 --- a/distribution/conf/nacos-logback.xml +++ b/distribution/conf/nacos-logback.xml @@ -624,7 +624,7 @@ - + diff --git a/naming/src/main/java/com/alibaba/nacos/naming/cluster/ServerListManager.java b/naming/src/main/java/com/alibaba/nacos/naming/cluster/ServerListManager.java index 5fe4fe38c..251e8ea7e 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/cluster/ServerListManager.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/cluster/ServerListManager.java @@ -24,7 +24,7 @@ import com.alibaba.nacos.core.cluster.MemberChangeListener; import com.alibaba.nacos.core.cluster.MemberMetaDataConstants; import com.alibaba.nacos.core.cluster.NodeState; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.consistency.persistent.raft.RaftPeer; import com.alibaba.nacos.naming.misc.GlobalExecutor; import com.alibaba.nacos.naming.misc.Loggers; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/DelegateConsistencyServiceImpl.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/DelegateConsistencyServiceImpl.java index d5402a620..d4d5e6294 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/DelegateConsistencyServiceImpl.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/DelegateConsistencyServiceImpl.java @@ -18,10 +18,7 @@ package com.alibaba.nacos.naming.consistency; import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.naming.consistency.ephemeral.EphemeralConsistencyService; -import com.alibaba.nacos.naming.consistency.persistent.ClusterVersionJudgement; -import com.alibaba.nacos.naming.consistency.persistent.PersistentConsistencyService; -import com.alibaba.nacos.naming.consistency.persistent.impl.PersistentServiceProcessor; -import com.alibaba.nacos.naming.consistency.persistent.raft.RaftConsistencyServiceImpl; +import com.alibaba.nacos.naming.consistency.persistent.PersistentConsistencyServiceDelegateImpl; import com.alibaba.nacos.naming.pojo.Record; import org.springframework.context.annotation.DependsOn; import org.springframework.stereotype.Service; @@ -36,29 +33,14 @@ import org.springframework.stereotype.Service; @Service("consistencyDelegate") public class DelegateConsistencyServiceImpl implements ConsistencyService { - private final ClusterVersionJudgement versionJudgement; - - private final RaftConsistencyServiceImpl oldPersistentConsistencyService; - - private final PersistentServiceProcessor newPersistentConsistencyService; + private final PersistentConsistencyServiceDelegateImpl persistentConsistencyService; private final EphemeralConsistencyService ephemeralConsistencyService; - private volatile boolean switchNewPersistentService = false; - - public DelegateConsistencyServiceImpl(ClusterVersionJudgement versionJudgement, - RaftConsistencyServiceImpl oldPersistentConsistencyService, - PersistentServiceProcessor newPersistentConsistencyService, + public DelegateConsistencyServiceImpl(PersistentConsistencyServiceDelegateImpl persistentConsistencyService, EphemeralConsistencyService ephemeralConsistencyService) { - this.versionJudgement = versionJudgement; - this.oldPersistentConsistencyService = oldPersistentConsistencyService; - this.newPersistentConsistencyService = newPersistentConsistencyService; + this.persistentConsistencyService = persistentConsistencyService; this.ephemeralConsistencyService = ephemeralConsistencyService; - this.init(); - } - - private void init() { - this.versionJudgement.registerObserver(isAllNewVersion -> switchNewPersistentService = isAllNewVersion, -1); } @Override @@ -81,8 +63,7 @@ public class DelegateConsistencyServiceImpl implements ConsistencyService { // this special key is listened by both: if (KeyBuilder.SERVICE_META_KEY_PREFIX.equals(key)) { - oldPersistentConsistencyService.listen(key, listener); - newPersistentConsistencyService.listen(key, listener); + persistentConsistencyService.listen(key, listener); ephemeralConsistencyService.listen(key, listener); return; } @@ -92,23 +73,15 @@ public class DelegateConsistencyServiceImpl implements ConsistencyService { @Override public void unListen(String key, RecordListener listener) throws NacosException { - ConsistencyService service = mapConsistencyService(key); - service.unListen(key, listener); - if (service instanceof PersistentConsistencyService && !switchNewPersistentService) { - newPersistentConsistencyService.unListen(key, listener); - } + mapConsistencyService(key).unListen(key, listener); } @Override public boolean isAvailable() { - return ephemeralConsistencyService.isAvailable() && switchOne().isAvailable(); + return ephemeralConsistencyService.isAvailable() && persistentConsistencyService.isAvailable(); } private ConsistencyService mapConsistencyService(String key) { - return KeyBuilder.matchEphemeralKey(key) ? ephemeralConsistencyService : switchOne(); - } - - private PersistentConsistencyService switchOne() { - return switchNewPersistentService ? newPersistentConsistencyService : oldPersistentConsistencyService; + return KeyBuilder.matchEphemeralKey(key) ? ephemeralConsistencyService : persistentConsistencyService; } } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/KeyBuilder.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/KeyBuilder.java index 092896612..41ce7cb12 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/KeyBuilder.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/KeyBuilder.java @@ -19,6 +19,8 @@ package com.alibaba.nacos.naming.consistency; import com.alibaba.nacos.naming.misc.UtilsAndCommons; import org.apache.commons.lang3.StringUtils; +import static com.alibaba.nacos.naming.misc.UtilsAndCommons.RAFT_CACHE_FILE_PREFIX; + /** * Key operations for data. * @@ -81,8 +83,7 @@ public class KeyBuilder { } public static boolean matchSwitchKey(String key) { - return key.endsWith(UtilsAndCommons.SWITCH_DOMAIN_NAME) || key - .endsWith(UtilsAndCommons.SWITCH_DOMAIN_NAME + UtilsAndCommons.RAFT_CACHE_FILE_SUFFIX); + return key.endsWith(UtilsAndCommons.SWITCH_DOMAIN_NAME); } public static boolean matchServiceName(String key, String namespaceId, String serviceName) { @@ -141,4 +142,8 @@ public class KeyBuilder { public static String getServiceName(String key) { return key.split(NAMESPACE_KEY_CONNECTOR)[1]; } + + public static boolean isDatumCacheFile(String key) { + return key.startsWith(RAFT_CACHE_FILE_PREFIX); + } } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroConsistencyServiceImpl.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroConsistencyServiceImpl.java index 7f679fec2..26cfac16d 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroConsistencyServiceImpl.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroConsistencyServiceImpl.java @@ -19,7 +19,6 @@ package com.alibaba.nacos.naming.consistency.ephemeral.distro; import com.alibaba.nacos.api.common.Constants; import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.common.utils.Objects; -import com.alibaba.nacos.core.utils.ApplicationUtils; import com.alibaba.nacos.naming.cluster.ServerStatus; import com.alibaba.nacos.naming.cluster.transport.Serializer; import com.alibaba.nacos.consistency.DataOperation; @@ -28,16 +27,10 @@ import com.alibaba.nacos.naming.consistency.KeyBuilder; import com.alibaba.nacos.naming.consistency.RecordListener; import com.alibaba.nacos.naming.consistency.ephemeral.EphemeralConsistencyService; import com.alibaba.nacos.naming.consistency.ephemeral.distro.combined.DistroHttpCombinedKey; -import com.alibaba.nacos.naming.consistency.ephemeral.distro.combined.DistroHttpCombinedKeyTaskFailedHandler; -import com.alibaba.nacos.naming.consistency.ephemeral.distro.combined.DistroHttpDelayTaskProcessor; -import com.alibaba.nacos.naming.consistency.ephemeral.distro.component.DistroDataStorageImpl; -import com.alibaba.nacos.naming.consistency.ephemeral.distro.component.DistroHttpAgent; import com.alibaba.nacos.core.distributed.distro.DistroProtocol; -import com.alibaba.nacos.core.distributed.distro.component.DistroComponentHolder; import com.alibaba.nacos.core.distributed.distro.component.DistroDataProcessor; import com.alibaba.nacos.core.distributed.distro.entity.DistroData; import com.alibaba.nacos.core.distributed.distro.entity.DistroKey; -import com.alibaba.nacos.core.distributed.distro.task.DistroTaskEngineHolder; import com.alibaba.nacos.naming.core.DistroMapper; import com.alibaba.nacos.naming.core.Instances; import com.alibaba.nacos.naming.core.Service; @@ -102,20 +95,6 @@ public class DistroConsistencyServiceImpl implements EphemeralConsistencyService this.switchDomain = switchDomain; this.globalConfig = globalConfig; this.distroProtocol = distroProtocol; - registerDistroComponent(); - } - - private void registerDistroComponent() { - DistroComponentHolder componentHolder = ApplicationUtils.getBean(DistroComponentHolder.class); - DistroTaskEngineHolder taskEngineHolder = ApplicationUtils.getBean(DistroTaskEngineHolder.class); - componentHolder.registerDataStorage(KeyBuilder.INSTANCE_LIST_KEY_PREFIX, - new DistroDataStorageImpl(dataStore, distroMapper)); - componentHolder.registerTransportAgent(KeyBuilder.INSTANCE_LIST_KEY_PREFIX, new DistroHttpAgent()); - componentHolder.registerFailedTaskHandler(KeyBuilder.INSTANCE_LIST_KEY_PREFIX, - new DistroHttpCombinedKeyTaskFailedHandler(globalConfig, taskEngineHolder)); - taskEngineHolder.registerNacosTaskProcessor(KeyBuilder.INSTANCE_LIST_KEY_PREFIX, - new DistroHttpDelayTaskProcessor(globalConfig, taskEngineHolder)); - componentHolder.registerDataProcessor(this); } @PostConstruct diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroHttpData.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroHttpData.java index 7f22ed4d3..314770313 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroHttpData.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroHttpData.java @@ -33,8 +33,8 @@ public class DistroHttpData extends DistroData { private Object deserializedContent; - public DistroHttpData(DistroKey distroKey, byte[] content, Object deserializedContent) { - super(distroKey, content); + public DistroHttpData(DistroKey distroKey, Object deserializedContent) { + setDistroKey(distroKey); this.deserializedContent = deserializedContent; } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroHttpRegistry.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroHttpRegistry.java new file mode 100644 index 000000000..16c81fdab --- /dev/null +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroHttpRegistry.java @@ -0,0 +1,77 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.naming.consistency.ephemeral.distro; + +import com.alibaba.nacos.core.distributed.distro.component.DistroComponentHolder; +import com.alibaba.nacos.core.distributed.distro.task.DistroTaskEngineHolder; +import com.alibaba.nacos.naming.consistency.KeyBuilder; +import com.alibaba.nacos.naming.consistency.ephemeral.distro.combined.DistroHttpCombinedKeyTaskFailedHandler; +import com.alibaba.nacos.naming.consistency.ephemeral.distro.combined.DistroHttpDelayTaskProcessor; +import com.alibaba.nacos.naming.consistency.ephemeral.distro.component.DistroDataStorageImpl; +import com.alibaba.nacos.naming.consistency.ephemeral.distro.component.DistroHttpAgent; +import com.alibaba.nacos.naming.core.DistroMapper; +import com.alibaba.nacos.naming.misc.GlobalConfig; +import org.springframework.stereotype.Component; + +import javax.annotation.PostConstruct; + +/** + * Distro http registry. + * + * @author xiweng.yy + */ +@Component +public class DistroHttpRegistry { + + private final DistroComponentHolder componentHolder; + + private final DistroTaskEngineHolder taskEngineHolder; + + private final DataStore dataStore; + + private final DistroMapper distroMapper; + + private final GlobalConfig globalConfig; + + private final DistroConsistencyServiceImpl consistencyService; + + public DistroHttpRegistry(DistroComponentHolder componentHolder, DistroTaskEngineHolder taskEngineHolder, + DataStore dataStore, DistroMapper distroMapper, GlobalConfig globalConfig, + DistroConsistencyServiceImpl consistencyService) { + this.componentHolder = componentHolder; + this.taskEngineHolder = taskEngineHolder; + this.dataStore = dataStore; + this.distroMapper = distroMapper; + this.globalConfig = globalConfig; + this.consistencyService = consistencyService; + } + + /** + * Register necessary component to distro protocol for HTTP implement. + */ + @PostConstruct + public void doRegister() { + componentHolder.registerDataStorage(KeyBuilder.INSTANCE_LIST_KEY_PREFIX, + new DistroDataStorageImpl(dataStore, distroMapper)); + componentHolder.registerTransportAgent(KeyBuilder.INSTANCE_LIST_KEY_PREFIX, new DistroHttpAgent()); + componentHolder.registerFailedTaskHandler(KeyBuilder.INSTANCE_LIST_KEY_PREFIX, + new DistroHttpCombinedKeyTaskFailedHandler(globalConfig, taskEngineHolder)); + taskEngineHolder.registerNacosTaskProcessor(KeyBuilder.INSTANCE_LIST_KEY_PREFIX, + new DistroHttpDelayTaskProcessor(globalConfig, taskEngineHolder)); + componentHolder.registerDataProcessor(consistencyService); + } +} diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/component/DistroDataStorageImpl.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/component/DistroDataStorageImpl.java index 111b5065b..645229588 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/component/DistroDataStorageImpl.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/component/DistroDataStorageImpl.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.naming.consistency.ephemeral.distro.component; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.cluster.transport.Serializer; import com.alibaba.nacos.naming.consistency.Datum; import com.alibaba.nacos.naming.consistency.KeyBuilder; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/PersistentConsistencyServiceDelegateImpl.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/PersistentConsistencyServiceDelegateImpl.java new file mode 100644 index 000000000..a22d22d26 --- /dev/null +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/PersistentConsistencyServiceDelegateImpl.java @@ -0,0 +1,91 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.naming.consistency.persistent; + +import com.alibaba.nacos.api.exception.NacosException; +import com.alibaba.nacos.naming.consistency.Datum; +import com.alibaba.nacos.naming.consistency.RecordListener; +import com.alibaba.nacos.naming.consistency.persistent.impl.PersistentServiceProcessor; +import com.alibaba.nacos.naming.consistency.persistent.raft.RaftConsistencyServiceImpl; +import com.alibaba.nacos.naming.pojo.Record; +import org.springframework.stereotype.Component; + +/** + * Persistent consistency service delegate. + * + * @author xiweng.yy + */ +@Component("persistentConsistencyServiceDelegate") +public class PersistentConsistencyServiceDelegateImpl implements PersistentConsistencyService { + + private final ClusterVersionJudgement versionJudgement; + + private final RaftConsistencyServiceImpl oldPersistentConsistencyService; + + private final PersistentServiceProcessor newPersistentConsistencyService; + + private volatile boolean switchNewPersistentService = false; + + public PersistentConsistencyServiceDelegateImpl(ClusterVersionJudgement versionJudgement, + RaftConsistencyServiceImpl oldPersistentConsistencyService, + PersistentServiceProcessor newPersistentConsistencyService) { + this.versionJudgement = versionJudgement; + this.oldPersistentConsistencyService = oldPersistentConsistencyService; + this.newPersistentConsistencyService = newPersistentConsistencyService; + init(); + } + + private void init() { + this.versionJudgement.registerObserver(isAllNewVersion -> switchNewPersistentService = isAllNewVersion, -1); + } + + @Override + public void put(String key, Record value) throws NacosException { + switchOne().put(key, value); + } + + @Override + public void remove(String key) throws NacosException { + switchOne().remove(key); + } + + @Override + public Datum get(String key) throws NacosException { + return switchOne().get(key); + } + + @Override + public void listen(String key, RecordListener listener) throws NacosException { + oldPersistentConsistencyService.listen(key, listener); + newPersistentConsistencyService.listen(key, listener); + } + + @Override + public void unListen(String key, RecordListener listener) throws NacosException { + newPersistentConsistencyService.unListen(key, listener); + oldPersistentConsistencyService.unListen(key, listener); + } + + @Override + public boolean isAvailable() { + return switchOne().isAvailable(); + } + + private PersistentConsistencyService switchOne() { + return switchNewPersistentService ? newPersistentConsistencyService : oldPersistentConsistencyService; + } +} diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/PersistentNotifier.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/PersistentNotifier.java index e54eae2de..e1be3ef5f 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/PersistentNotifier.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/PersistentNotifier.java @@ -48,7 +48,7 @@ public final class PersistentNotifier extends Subscriber { /** * register listener with key. * - * @param key key + * @param key key * @param listener {@link RecordListener} */ public void registerListener(final String key, final RecordListener listener) { @@ -59,7 +59,7 @@ public final class PersistentNotifier extends Subscriber { /** * deregister listener by key. * - * @param key key + * @param key key * @param listener {@link RecordListener} */ public void deregisterListener(final String key, final RecordListener listener) { @@ -70,12 +70,25 @@ public final class PersistentNotifier extends Subscriber { } /** - * notify value to listener with {@link DataOperation} and key. + * deregister all listener by key. * * @param key key + */ + public void deregisterAllListener(final String key) { + listenerMap.remove(key); + } + + public Map> getListeners() { + return listenerMap; + } + + /** + * notify value to listener with {@link DataOperation} and key. + * + * @param key key * @param action {@link DataOperation} - * @param value value - * @param type + * @param value value + * @param type */ public void notify(final String key, final DataOperation action, final T value) { if (listenerMap.containsKey(KeyBuilder.SERVICE_META_KEY_PREFIX)) { @@ -89,18 +102,16 @@ public final class PersistentNotifier extends Subscriber { listener.onDelete(key); } } catch (Throwable e) { - Loggers.RAFT - .error("[NACOS-RAFT] error while notifying listener of key: {}", key, - e); + Loggers.RAFT.error("[NACOS-RAFT] error while notifying listener of key: {}", key, e); } } } } - + if (!listenerMap.containsKey(key)) { return; } - + for (RecordListener listener : listenerMap.get(key)) { try { if (action == DataOperation.CHANGE) { diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingKvStorage.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingKvStorage.java new file mode 100644 index 000000000..86ff664d8 --- /dev/null +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingKvStorage.java @@ -0,0 +1,229 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.naming.consistency.persistent.impl; + +import com.alibaba.nacos.api.exception.NacosException; +import com.alibaba.nacos.api.exception.runtime.NacosRuntimeException; +import com.alibaba.nacos.common.utils.StringUtils; +import com.alibaba.nacos.core.exception.ErrorCode; +import com.alibaba.nacos.core.exception.KvStorageException; +import com.alibaba.nacos.core.storage.StorageFactory; +import com.alibaba.nacos.core.storage.kv.KvStorage; +import com.alibaba.nacos.core.storage.kv.MemoryKvStorage; +import com.alibaba.nacos.core.utils.TimerContext; +import com.alibaba.nacos.naming.consistency.KeyBuilder; +import com.alibaba.nacos.naming.misc.Loggers; + +import java.io.File; +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; +import java.util.function.Function; + +/** + * Kv storage implementation for naming. + * + * @author xiweng.yy + */ +public class NamingKvStorage extends MemoryKvStorage { + + private final String baseDir; + + private final KvStorage baseDirStorage; + + private final Map namespaceKvStorage; + + public NamingKvStorage(final String baseDir) throws Exception { + this.baseDir = baseDir; + this.baseDirStorage = StorageFactory.createKvStorage(KvStorage.KvType.File, "naming-persistent", baseDir); + this.namespaceKvStorage = new ConcurrentHashMap<>(16); + } + + @Override + public byte[] get(byte[] key) throws KvStorageException { + // First get the data from the memory Cache + byte[] result = super.get(key); + if (null == result) { + try { + KvStorage storage = createActualStorageIfAbsent(key); + result = null == storage ? null : storage.get(key); + if (null != result) { + super.put(key, result); + } + } catch (Exception e) { + throw new KvStorageException(ErrorCode.KVStorageWriteError.getCode(), + "Get data failed, key: " + new String(key), e); + } + } + return result; + } + + @Override + public Map batchGet(List keys) throws KvStorageException { + Map result = new HashMap<>(keys.size()); + for (byte[] key : keys) { + byte[] val = get(key); + if (val != null) { + result.put(key, val); + } + } + return result; + } + + @Override + public void put(byte[] key, byte[] value) throws KvStorageException { + try { + KvStorage storage = createActualStorageIfAbsent(key); + storage.put(key, value); + } catch (Exception e) { + throw new KvStorageException(ErrorCode.KVStorageWriteError.getCode(), + "Put data failed, key: " + new String(key), e); + } + // after actual storage put success, put it in memory, memory put should success all the time + super.put(key, value); + } + + @Override + public void batchPut(List keys, List values) throws KvStorageException { + if (keys.size() != values.size()) { + throw new KvStorageException(ErrorCode.KVStorageBatchWriteError, + "key's size must be equal to value's size"); + } + int size = keys.size(); + for (int i = 0; i < size; i++) { + put(keys.get(i), values.get(i)); + } + } + + @Override + public void delete(byte[] key) throws KvStorageException { + try { + KvStorage storage = createActualStorageIfAbsent(key); + if (null != storage) { + storage.delete(key); + } + } catch (Exception e) { + throw new KvStorageException(ErrorCode.KVStorageDeleteError.getCode(), + "Delete data failed, key: " + new String(key), e); + } + // after actual storage delete success, put it in memory, memory delete should success all the time + super.delete(key); + } + + @Override + public void batchDelete(List key) throws KvStorageException { + for (byte[] each : key) { + delete(each); + } + } + + @Override + public void doSnapshot(String backupPath) throws KvStorageException { + baseDirStorage.doSnapshot(backupPath); + } + + @Override + public void snapshotLoad(String path) throws KvStorageException { + final KvStorageException e = TimerContext.run(() -> { + try { + baseDirStorage.snapshotLoad(path); + loadSnapshotFromActualStorage(baseDirStorage); + loadNamespaceSnapshot(); + return null; + } catch (KvStorageException exception) { + return exception; + } + }, "naming kv storage load snapshot", Loggers.RAFT); + if (e != null) { + throw e; + } + } + + private void loadSnapshotFromActualStorage(KvStorage actualStorage) throws KvStorageException { + for (byte[] each : actualStorage.allKeys()) { + byte[] datum = actualStorage.get(each); + super.put(each, datum); + } + } + + private void loadNamespaceSnapshot() { + for (String each : getAllNamespaceDirs()) { + try { + KvStorage kvStorage = createActualStorageIfAbsent(each); + loadSnapshotFromActualStorage(kvStorage); + } catch (Exception e) { + Loggers.RAFT.error("load snapshot for namespace {} failed", each, e); + } + } + } + + private List getAllNamespaceDirs() { + File[] files = new File(baseDir).listFiles(); + List result = Collections.emptyList(); + if (null != files) { + result = new ArrayList<>(files.length); + for (File each : files) { + if (each.isDirectory()) { + result.add(each.getName()); + } + } + } + return Collections.unmodifiableList(result); + } + + @Override + public List allKeys() throws KvStorageException { + return super.allKeys(); + } + + @Override + public void shutdown() { + baseDirStorage.shutdown(); + for (KvStorage each : namespaceKvStorage.values()) { + each.shutdown(); + } + namespaceKvStorage.clear(); + super.shutdown(); + } + + private KvStorage createActualStorageIfAbsent(byte[] key) throws Exception { + String keyString = new String(key); + String namespace = KeyBuilder.getNamespace(keyString); + return createActualStorageIfAbsent(namespace); + } + + private KvStorage createActualStorageIfAbsent(String namespace) throws Exception { + if (StringUtils.isBlank(namespace)) { + return baseDirStorage; + } + + Function kvStorageBuilder = key -> { + try { + String namespacePath = Paths.get(baseDir, key).toString(); + return StorageFactory.createKvStorage(KvType.File, "naming-persistent", namespacePath); + } catch (Exception e) { + throw new NacosRuntimeException(NacosException.SERVER_ERROR, e); + } + }; + namespaceKvStorage.computeIfAbsent(namespace, kvStorageBuilder); + return namespaceKvStorage.get(namespace); + } +} diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingSnapshotOperation.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingSnapshotOperation.java index 9653a04a7..108b25de9 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingSnapshotOperation.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingSnapshotOperation.java @@ -23,9 +23,9 @@ import com.alibaba.nacos.consistency.snapshot.SnapshotOperation; import com.alibaba.nacos.consistency.snapshot.Writer; import com.alibaba.nacos.core.distributed.raft.utils.RaftExecutor; import com.alibaba.nacos.core.storage.kv.KvStorage; -import com.alibaba.nacos.core.utils.DiskUtils; import com.alibaba.nacos.core.utils.TimerContext; import com.alibaba.nacos.naming.misc.Loggers; +import com.alibaba.nacos.sys.utils.DiskUtils; import com.alipay.sofa.jraft.util.CRC64; import java.nio.file.Paths; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/PersistentServiceProcessor.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/PersistentServiceProcessor.java index 035e8d377..1d807dcfd 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/PersistentServiceProcessor.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/impl/PersistentServiceProcessor.java @@ -34,30 +34,30 @@ import com.alibaba.nacos.consistency.snapshot.SnapshotOperation; import com.alibaba.nacos.core.distributed.ProtocolManager; import com.alibaba.nacos.core.exception.ErrorCode; import com.alibaba.nacos.core.exception.KvStorageException; -import com.alibaba.nacos.core.storage.StorageFactory; import com.alibaba.nacos.core.storage.kv.KvStorage; -import com.alibaba.nacos.core.utils.ApplicationUtils; import com.alibaba.nacos.naming.consistency.Datum; +import com.alibaba.nacos.naming.consistency.KeyBuilder; import com.alibaba.nacos.naming.consistency.RecordListener; import com.alibaba.nacos.naming.consistency.ValueChangeEvent; import com.alibaba.nacos.naming.consistency.persistent.ClusterVersionJudgement; import com.alibaba.nacos.naming.consistency.persistent.PersistentConsistencyService; import com.alibaba.nacos.naming.consistency.persistent.PersistentNotifier; -import com.alibaba.nacos.naming.consistency.persistent.raft.RaftStore; import com.alibaba.nacos.naming.misc.Loggers; import com.alibaba.nacos.naming.misc.UtilsAndCommons; import com.alibaba.nacos.naming.pojo.Record; import com.alibaba.nacos.naming.utils.Constants; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.google.protobuf.ByteString; +import org.apache.commons.lang3.reflect.TypeUtils; import org.springframework.stereotype.Service; +import java.lang.reflect.Type; import java.nio.file.Paths; import java.util.ArrayList; import java.util.Collections; -import java.util.HashMap; import java.util.List; import java.util.Map; -import java.util.concurrent.CompletableFuture; +import java.util.concurrent.TimeUnit; import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantReadWriteLock; @@ -97,11 +97,9 @@ public class PersistentServiceProcessor extends LogProcessor4CP implements Persi private final KvStorage kvStorage; - private final RaftStore oldStore; - private final ClusterVersionJudgement versionJudgement; - private final Serializer serializer = SerializeFactory.getDefault(); + private final Serializer serializer; /** * During snapshot processing, the processing of other requests needs to be paused. @@ -122,17 +120,22 @@ public class PersistentServiceProcessor extends LogProcessor4CP implements Persi */ private volatile boolean hasError = false; + /** + * If use old raft, should not notify listener even new listener add. + */ + private volatile boolean startNotify = false; + public PersistentServiceProcessor(final ProtocolManager protocolManager, - final ClusterVersionJudgement versionJudgement, final RaftStore oldStore) throws Exception { + final ClusterVersionJudgement versionJudgement) throws Exception { this.protocol = protocolManager.getCpProtocol(); - this.oldStore = oldStore; this.versionJudgement = versionJudgement; - this.kvStorage = StorageFactory.createKvStorage(KvStorage.KvType.File, "naming-persistent", - Paths.get(UtilsAndCommons.DATA_BASE_DIR, "persistent").toString()); + this.kvStorage = new NamingKvStorage(Paths.get(UtilsAndCommons.DATA_BASE_DIR, "data").toString()); + this.serializer = SerializeFactory.getSerializer("JSON"); this.notifier = new PersistentNotifier(key -> { try { byte[] data = kvStorage.get(ByteUtils.toBytes(key)); - return serializer.deserialize(data); + Datum datum = serializer.deserialize(data, getDatumTypeFromKey(key)); + return null != datum ? datum.value : null; } catch (KvStorageException ex) { throw new NacosRuntimeException(ex.getErrCode(), ex.getErrMsg()); } @@ -150,19 +153,32 @@ public class PersistentServiceProcessor extends LogProcessor4CP implements Persi // If you choose to use the new RAFT protocol directly, there will be no compatible logical execution if (ApplicationUtils.getProperty(Constants.NACOS_NAMING_USE_NEW_RAFT_FIRST, Boolean.class, false)) { NotifyCenter.registerSubscriber(notifier); + waitLeader(); + startNotify = true; } else { this.versionJudgement.registerObserver(isNewVersion -> { if (isNewVersion) { - loadFromOldData(); NotifyCenter.registerSubscriber(notifier); + startNotify = true; } }, 10); } } + private void waitLeader() { + while (!hasLeader && !hasError) { + Loggers.RAFT.info("Waiting Jraft leader vote ..."); + try { + TimeUnit.MILLISECONDS.sleep(500); + } catch (InterruptedException ignored) { + } + } + } + @Override public Response onRequest(GetRequest request) { - final List keys = serializer.deserialize(request.getData().toByteArray(), List.class); + final List keys = serializer + .deserialize(request.getData().toByteArray(), TypeUtils.parameterize(List.class, byte[].class)); final Lock lock = readLock; lock.lock(); try { @@ -207,10 +223,11 @@ public class PersistentServiceProcessor extends LogProcessor4CP implements Persi private void publishValueChangeEvent(final Op op, final BatchWriteRequest request) { final List keys = request.getKeys(); - final List values = request.getKeys(); + final List values = request.getValues(); for (int i = 0; i < keys.size(); i++) { final String key = new String(keys.get(i)); - final Record value = serializer.deserialize(values.get(i)); + final Datum datum = serializer.deserialize(values.get(i), getDatumTypeFromKey(key)); + final Record value = null != datum ? datum.value : null; final ValueChangeEvent event = ValueChangeEvent.builder().key(key).value(value) .action(Op.Delete.equals(op) ? DataOperation.DELETE : DataOperation.CHANGE).build(); NotifyCenter.publishEvent(event); @@ -227,62 +244,11 @@ public class PersistentServiceProcessor extends LogProcessor4CP implements Persi return Collections.singletonList(new NamingSnapshotOperation(this.kvStorage, lock)); } - /** - * Pull old data into the new data store. When loading old data information, write locks must be added, and new - * requests can be processed only after the old data has been loaded - */ - @SuppressWarnings("unchecked") - public void loadFromOldData() { - final Lock lock = this.lock.writeLock(); - lock.lock(); - Loggers.RAFT.warn("start to load data to new raft protocol!!!"); - try { - if (protocol.isLeader(Constants.NAMING_PERSISTENT_SERVICE_GROUP)) { - Map datumMap = new HashMap<>(64); - oldStore.loadDatums(null, datumMap); - int totalSize = datumMap.size(); - List keys = new ArrayList<>(totalSize); - List values = new ArrayList<>(totalSize); - int batchSize = 100; - List futures = new ArrayList<>(16); - for (Map.Entry entry : datumMap.entrySet()) { - totalSize--; - keys.add(ByteUtils.toBytes(entry.getKey())); - values.add(serializer.serialize(entry.getValue().value)); - if (keys.size() == batchSize || totalSize == 0) { - BatchWriteRequest request = new BatchWriteRequest(); - request.setKeys(keys); - request.setValues(values); - CompletableFuture future = protocol.submitAsync( - Log.newBuilder().setGroup(Constants.NAMING_PERSISTENT_SERVICE_GROUP) - .setOperation(Op.Write.name()) - .setData(ByteString.copyFrom(serializer.serialize(request))).build()) - .whenComplete(((response, throwable) -> { - if (throwable == null) { - Loggers.RAFT.error("submit old raft data result : {}", response); - } else { - Loggers.RAFT.error("submit old raft data occur exception : {}", throwable); - } - })); - futures.add(future); - keys.clear(); - values.clear(); - } - } - CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])); - } - } catch (Throwable ex) { - hasError = true; - Loggers.RAFT.error("load old raft data occur exception : {}", ex); - } finally { - lock.unlock(); - } - } - @Override public void put(String key, Record value) throws NacosException { final BatchWriteRequest req = new BatchWriteRequest(); - req.append(ByteUtils.toBytes(key), serializer.serialize(value)); + Datum datum = Datum.createDatum(key, value); + req.append(ByteUtils.toBytes(key), serializer.serialize(datum)); final Log log = Log.newBuilder().setData(ByteString.copyFrom(serializer.serialize(req))) .setGroup(Constants.NAMING_PERSISTENT_SERVICE_GROUP).setOperation(Op.Write.desc).build(); try { @@ -317,8 +283,7 @@ public class PersistentServiceProcessor extends LogProcessor4CP implements Persi BatchReadResponse response = serializer .deserialize(resp.getData().toByteArray(), BatchReadResponse.class); final List rValues = response.getValues(); - Record record = serializer.deserialize(rValues.get(0)); - return Datum.createDatum(key, record); + return rValues.isEmpty() ? null : serializer.deserialize(rValues.get(0), getDatumTypeFromKey(key)); } throw new NacosException(ErrorCode.ProtoReadError.getCode(), resp.getErrMsg()); } catch (Throwable e) { @@ -329,6 +294,9 @@ public class PersistentServiceProcessor extends LogProcessor4CP implements Persi @Override public void listen(String key, RecordListener listener) throws NacosException { notifier.registerListener(key, listener); + if (startNotify) { + notifierDatumIfAbsent(key, listener); + } } @Override @@ -346,4 +314,53 @@ public class PersistentServiceProcessor extends LogProcessor4CP implements Persi public boolean isAvailable() { return hasLeader && !hasError; } + + private Type getDatumTypeFromKey(String key) { + return TypeUtils.parameterize(Datum.class, getClassOfRecordFromKey(key)); + } + + private Class getClassOfRecordFromKey(String key) { + if (KeyBuilder.matchSwitchKey(key)) { + return com.alibaba.nacos.naming.misc.SwitchDomain.class; + } else if (KeyBuilder.matchServiceMetaKey(key)) { + return com.alibaba.nacos.naming.core.Service.class; + } else if (KeyBuilder.matchInstanceListKey(key)) { + return com.alibaba.nacos.naming.core.Instances.class; + } + return Record.class; + } + + private void notifierDatumIfAbsent(String key, RecordListener listener) throws NacosException { + if (KeyBuilder.SERVICE_META_KEY_PREFIX.equals(key)) { + notifierAllServiceMeta(listener); + } else { + Datum datum = get(key); + if (null != datum) { + notifierDatum(key, datum, listener); + } + } + } + + /** + * This notify should only notify once during startup. See {@link com.alibaba.nacos.naming.core.ServiceManager#init()} + */ + private void notifierAllServiceMeta(RecordListener listener) throws NacosException { + for (byte[] each : kvStorage.allKeys()) { + String key = new String(each); + if (listener.interests(key)) { + Datum datum = get(key); + if (null != datum) { + notifierDatum(key, datum, listener); + } + } + } + } + + private void notifierDatum(String key, Datum datum, RecordListener listener) { + try { + listener.onChange(key, datum.value); + } catch (Exception e) { + Loggers.RAFT.error("NACOS-RAFT failed to notify listener", e); + } + } } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftConsistencyServiceImpl.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftConsistencyServiceImpl.java index d7e118846..1643f216a 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftConsistencyServiceImpl.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftConsistencyServiceImpl.java @@ -18,7 +18,7 @@ package com.alibaba.nacos.naming.consistency.persistent.raft; import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.api.exception.runtime.NacosRuntimeException; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils;; import com.alibaba.nacos.naming.cluster.ServerStatus; import com.alibaba.nacos.naming.consistency.Datum; import com.alibaba.nacos.naming.consistency.KeyBuilder; @@ -95,14 +95,11 @@ public class RaftConsistencyServiceImpl implements PersistentConsistencyService checkIsStopWork(); try { if (KeyBuilder.matchInstanceListKey(key) && !raftCore.isLeader()) { - Datum datum = new Datum(); - datum.key = key; - raftCore.onDelete(datum.key, peers.getLeader()); - raftCore.unlistenAll(key); - return; + raftCore.onDelete(key, peers.getLeader()); + } else { + raftCore.signalDelete(key); } - raftCore.signalDelete(key); - raftCore.unlistenAll(key); + raftCore.unListenAll(key); } catch (Exception e) { Loggers.RAFT.error("Raft remove failed.", e); throw new NacosException(NacosException.SERVER_ERROR, "Raft remove failed, key:" + key, e); @@ -117,7 +114,6 @@ public class RaftConsistencyServiceImpl implements PersistentConsistencyService @Override public void listen(String key, RecordListener listener) throws NacosException { - checkIsStopWork(); raftCore.listen(key, listener); } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftCore.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftCore.java index c9d995cb9..24021a834 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftCore.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftCore.java @@ -18,13 +18,14 @@ package com.alibaba.nacos.naming.consistency.persistent.raft; import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.api.exception.runtime.NacosRuntimeException; +import com.alibaba.nacos.common.http.Callback; import com.alibaba.nacos.common.lifecycle.Closeable; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.notify.EventPublisher; import com.alibaba.nacos.common.notify.NotifyCenter; +import com.alibaba.nacos.common.utils.ConcurrentHashSet; import com.alibaba.nacos.common.utils.JacksonUtils; -import com.alibaba.nacos.common.utils.ThreadUtils; import com.alibaba.nacos.consistency.DataOperation; -import com.alibaba.nacos.core.utils.ApplicationUtils; import com.alibaba.nacos.naming.consistency.Datum; import com.alibaba.nacos.naming.consistency.KeyBuilder; import com.alibaba.nacos.naming.consistency.RecordListener; @@ -42,12 +43,11 @@ import com.alibaba.nacos.naming.misc.SwitchDomain; import com.alibaba.nacos.naming.misc.UtilsAndCommons; import com.alibaba.nacos.naming.monitor.MetricsMonitor; import com.alibaba.nacos.naming.pojo.Record; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.fasterxml.jackson.core.type.TypeReference; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ArrayNode; import com.fasterxml.jackson.databind.node.ObjectNode; -import com.ning.http.client.AsyncCompletionHandler; -import com.ning.http.client.Response; import org.apache.commons.lang3.StringUtils; import org.apache.commons.lang3.math.NumberUtils; import org.springframework.context.annotation.DependsOn; @@ -58,7 +58,6 @@ import javax.annotation.PostConstruct; import java.io.ByteArrayOutputStream; import java.io.IOException; import java.io.UnsupportedEncodingException; -import java.net.HttpURLConnection; import java.net.URLDecoder; import java.net.URLEncoder; import java.nio.charset.StandardCharsets; @@ -69,7 +68,6 @@ import java.util.List; import java.util.Map; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentMap; -import java.util.concurrent.CopyOnWriteArrayList; import java.util.concurrent.CountDownLatch; import java.util.concurrent.ScheduledFuture; import java.util.concurrent.TimeUnit; @@ -80,8 +78,8 @@ import java.util.zip.GZIPOutputStream; /** * Raft core code. * - * @deprecated will remove in 1.4.x * @author nacos + * @deprecated will remove in 1.4.x */ @Deprecated @DependsOn("ProtocolManager") @@ -164,20 +162,13 @@ public class RaftCore implements Closeable { Loggers.RAFT.info("cache loaded, datum count: {}, current term: {}", datums.size(), peers.getTerm()); - while (true) { - if (publisher.currentEventSize() <= 0) { - break; - } - ThreadUtils.sleep(1000L); - } - initialized = true; Loggers.RAFT.info("finish to load data from disk, cost: {} ms.", (System.currentTimeMillis() - start)); masterTask = GlobalExecutor.registerMasterElection(new MasterElection()); heartbeatTask = GlobalExecutor.registerHeartbeat(new HeartBeat()); - + versionJudgement.registerObserver(isAllNewVersion -> { stopWork = isAllNewVersion; if (stopWork) { @@ -188,15 +179,15 @@ public class RaftCore implements Closeable { } } }, 100); - + NotifyCenter.registerSubscriber(notifier); - + Loggers.RAFT.info("timer started: leader timeout ms: {}, heart-beat timeout ms: {}", GlobalExecutor.LEADER_TIMEOUT_MS, GlobalExecutor.HEARTBEAT_INTERVAL_MS); } - public Map> getListeners() { - return listeners; + public Map> getListeners() { + return notifier.getListeners(); } /** @@ -222,7 +213,7 @@ public class RaftCore implements Closeable { raftProxy.proxyPostLarge(leader.ip, API_PUB, params.toString(), parameters); return; } - + OPERATE_LOCK.lock(); try { final long start = System.currentTimeMillis(); @@ -250,25 +241,28 @@ public class RaftCore implements Closeable { continue; } final String url = buildUrl(server, API_ON_PUB); - HttpClient.asyncHttpPostLarge(url, Arrays.asList("key=" + key), content, - new AsyncCompletionHandler() { - @Override - public Integer onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { - Loggers.RAFT - .warn("[RAFT] failed to publish data to peer, datumId={}, peer={}, http code={}", - datum.key, server, response.getStatusCode()); - return 1; - } - latch.countDown(); - return 0; - } - - @Override - public STATE onContentWriteCompleted() { - return STATE.CONTINUE; - } - }); + HttpClient.asyncHttpPostLarge(url, Arrays.asList("key=" + key), content, new Callback() { + @Override + public void onReceive(RestResult result) { + if (!result.ok()) { + Loggers.RAFT + .warn("[RAFT] failed to publish data to peer, datumId={}, peer={}, http code={}", + datum.key, server, result.getCode()); + return; + } + latch.countDown(); + } + + @Override + public void onError(Throwable throwable) { + Loggers.RAFT.error("[RAFT] failed to publish data to peer", throwable); + } + + @Override + public void onCancel() { + + } + }); } @@ -316,21 +310,29 @@ public class RaftCore implements Closeable { for (final String server : peers.allServersWithoutMySelf()) { String url = buildUrl(server, API_ON_DEL); - HttpClient.asyncHttpDeleteLarge(url, null, json.toString(), new AsyncCompletionHandler() { + HttpClient.asyncHttpDeleteLarge(url, null, json.toString(), new Callback() { @Override - public Integer onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { + public void onReceive(RestResult result) { + if (!result.ok()) { Loggers.RAFT .warn("[RAFT] failed to delete data from peer, datumId={}, peer={}, http code={}", - key, server, response.getStatusCode()); - return 1; + key, server, result.getCode()); + return; } RaftPeer local = peers.local(); local.resetLeaderDue(); - - return 0; + } + + @Override + public void onError(Throwable throwable) { + Loggers.RAFT.error("[RAFT] failed to delete data from peer", throwable); + } + + @Override + public void onCancel() { + } }); } @@ -457,6 +459,8 @@ public class RaftCore implements Closeable { masterTask.cancel(true); Loggers.RAFT.warn("stop old raft protocol task for heartbeat task"); heartbeatTask.cancel(true); + Loggers.RAFT.warn("clean old cache datum for old raft"); + datums.clear(); } public class MasterElection implements Runnable { @@ -470,18 +474,18 @@ public class RaftCore implements Closeable { if (!peers.isReady()) { return; } - + RaftPeer local = peers.local(); local.leaderDueMs -= GlobalExecutor.TICK_PERIOD_MS; - + if (local.leaderDueMs > 0) { return; } - + // reset timeout local.resetLeaderDue(); local.resetHeartbeatDue(); - + sendVote(); } catch (Exception e) { Loggers.RAFT.warn("[RAFT] error while master election {}", e); @@ -506,22 +510,30 @@ public class RaftCore implements Closeable { for (final String server : peers.allServersWithoutMySelf()) { final String url = buildUrl(server, API_VOTE); try { - HttpClient.asyncHttpPost(url, null, params, new AsyncCompletionHandler() { + HttpClient.asyncHttpPost(url, null, params, new Callback() { @Override - public Integer onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { - Loggers.RAFT - .error("NACOS-RAFT vote failed: {}, url: {}", response.getResponseBody(), url); - return 1; + public void onReceive(RestResult result) { + if (!result.ok()) { + Loggers.RAFT.error("NACOS-RAFT vote failed: {}, url: {}", result.getCode(), url); + return; } - RaftPeer peer = JacksonUtils.toObj(response.getResponseBody(), RaftPeer.class); + RaftPeer peer = JacksonUtils.toObj(result.getData(), RaftPeer.class); Loggers.RAFT.info("received approve from peer: {}", JacksonUtils.toJson(peer)); peers.decideLeader(peer); - return 0; + } + + @Override + public void onError(Throwable throwable) { + Loggers.RAFT.error("error while sending vote to server: {}", server, throwable); + } + + @Override + public void onCancel() { + } }); } catch (Exception e) { @@ -658,28 +670,32 @@ public class RaftCore implements Closeable { if (Loggers.RAFT.isDebugEnabled()) { Loggers.RAFT.debug("send beat to server " + server); } - HttpClient.asyncHttpPostLarge(url, null, compressedBytes, new AsyncCompletionHandler() { + HttpClient.asyncHttpPostLarge(url, null, compressedBytes, new Callback() { @Override - public Integer onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { - Loggers.RAFT.error("NACOS-RAFT beat failed: {}, peer: {}", response.getResponseBody(), - server); + public void onReceive(RestResult result) { + if (!result.ok()) { + Loggers.RAFT.error("NACOS-RAFT beat failed: {}, peer: {}", result.getCode(), server); MetricsMonitor.getLeaderSendBeatFailedException().increment(); - return 1; + return; } - peers.update(JacksonUtils.toObj(response.getResponseBody(), RaftPeer.class)); + peers.update(JacksonUtils.toObj(result.getData(), RaftPeer.class)); if (Loggers.RAFT.isDebugEnabled()) { Loggers.RAFT.debug("receive beat response from: {}", url); } - return 0; } @Override - public void onThrowable(Throwable t) { - Loggers.RAFT.error("NACOS-RAFT error while sending heart-beat to peer: {} {}", server, t); + public void onError(Throwable throwable) { + Loggers.RAFT.error("NACOS-RAFT error while sending heart-beat to peer: {} {}", server, + throwable); MetricsMonitor.getLeaderSendBeatFailedException().increment(); } + + @Override + public void onCancel() { + + } }); } catch (Exception e) { Loggers.RAFT.error("error while sending heart-beat to peer: {} {}", server, e); @@ -802,15 +818,15 @@ public class RaftCore implements Closeable { // update datum entry String url = buildUrl(remote.ip, API_GET) + "?keys=" + URLEncoder.encode(keys, "UTF-8"); - HttpClient.asyncHttpGet(url, null, null, new AsyncCompletionHandler() { + HttpClient.asyncHttpGet(url, null, null, new Callback() { @Override - public Integer onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { - return 1; + public void onReceive(RestResult result) { + if (!result.ok()) { + return; } List datumList = JacksonUtils - .toObj(response.getResponseBody(), new TypeReference>() { + .toObj(result.getData(), new TypeReference>() { }); for (JsonNode datumJson : datumList) { @@ -855,8 +871,7 @@ public class RaftCore implements Closeable { raftStore.write(newDatum); datums.put(newDatum.key, newDatum); - - NotifyCenter.publishEvent(ValueChangeEvent.builder().key(newDatum.key).action(DataOperation.CHANGE).build()); + notifier.notify(newDatum.key, DataOperation.CHANGE, newDatum.value); local.resetLeaderDue(); @@ -880,9 +895,24 @@ public class RaftCore implements Closeable { OPERATE_LOCK.unlock(); } } - TimeUnit.MILLISECONDS.sleep(200); - return 0; + try { + TimeUnit.MILLISECONDS.sleep(200); + } catch (InterruptedException e) { + Loggers.RAFT.error("[RAFT-BEAT] Interrupted error ", e); + } + return; } + + @Override + public void onError(Throwable throwable) { + Loggers.RAFT.error("[RAFT-BEAT] failed to sync datum from leader", throwable); + } + + @Override + public void onCancel() { + + } + }); batch.clear(); @@ -920,21 +950,9 @@ public class RaftCore implements Closeable { * @param listener new listener */ public void listen(String key, RecordListener listener) { - - List listenerList = listeners.get(key); - if (listenerList != null && listenerList.contains(listener)) { - return; - } - - if (listenerList == null) { - listenerList = new CopyOnWriteArrayList<>(); - listeners.put(key, listenerList); - } + notifier.registerListener(key, listener); Loggers.RAFT.info("add listener: {}", key); - - listenerList.add(listener); - // if data present, notify immediately for (Datum datum : datums.values()) { if (!listener.interests(datum.key)) { @@ -956,22 +974,11 @@ public class RaftCore implements Closeable { * @param listener listener */ public void unListen(String key, RecordListener listener) { - - if (!listeners.containsKey(key)) { - return; - } - - for (RecordListener dl : listeners.get(key)) { - // TODO maybe use equal: - if (dl == listener) { - listeners.get(key).remove(listener); - break; - } - } + notifier.deregisterListener(key, listener); } - public void unlistenAll(String key) { - listeners.remove(key); + public void unListenAll(String key) { + notifier.deregisterAllListener(key); } public void setTerm(long term) { @@ -1055,7 +1062,9 @@ public class RaftCore implements Closeable { raftStore.delete(deleted); Loggers.RAFT.info("datum deleted, key: {}", key); } - NotifyCenter.publishEvent(ValueChangeEvent.builder().key(URLDecoder.decode(key, "UTF-8")).action(DataOperation.DELETE).build()); + NotifyCenter.publishEvent( + ValueChangeEvent.builder().key(URLDecoder.decode(key, "UTF-8")).action(DataOperation.DELETE) + .build()); } catch (UnsupportedEncodingException e) { Loggers.RAFT.warn("datum key decode failed: {}", key); } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftPeerSet.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftPeerSet.java index 10604779e..9cbeba813 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftPeerSet.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftPeerSet.java @@ -17,19 +17,19 @@ package com.alibaba.nacos.naming.consistency.persistent.raft; import com.alibaba.nacos.api.exception.NacosException; +import com.alibaba.nacos.common.http.Callback; import com.alibaba.nacos.common.lifecycle.Closeable; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.notify.NotifyCenter; import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.MemberChangeListener; import com.alibaba.nacos.core.cluster.MembersChangeEvent; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; import com.alibaba.nacos.naming.misc.HttpClient; import com.alibaba.nacos.naming.misc.Loggers; import com.alibaba.nacos.naming.misc.NetUtils; -import com.ning.http.client.AsyncCompletionHandler; -import com.ning.http.client.Response; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.collections.SortedBag; import org.apache.commons.collections.bag.TreeBag; import org.apache.commons.lang3.StringUtils; @@ -37,7 +37,6 @@ import org.springframework.context.annotation.DependsOn; import org.springframework.stereotype.Component; import javax.annotation.PostConstruct; -import java.net.HttpURLConnection; import java.util.Arrays; import java.util.Collection; import java.util.HashMap; @@ -51,8 +50,8 @@ import java.util.concurrent.atomic.AtomicLong; /** * Sets of raft peers. * - * @deprecated will remove in 1.4.x * @author nacos + * @deprecated will remove in 1.4.x */ @Deprecated @Component @@ -71,7 +70,7 @@ public class RaftPeerSet extends MemberChangeListener implements Closeable { private volatile boolean ready = false; - private Set oldMembers; + private Set oldMembers = new HashSet<>(); public RaftPeerSet(ServerMemberManager memberManager) { this.memberManager = memberManager; @@ -233,20 +232,27 @@ public class RaftPeerSet extends MemberChangeListener implements Closeable { if (!Objects.equals(peer, candidate) && peer.state == RaftPeer.State.LEADER) { try { String url = RaftCore.buildUrl(peer.ip, RaftCore.API_GET_PEER); - HttpClient.asyncHttpGet(url, null, params, new AsyncCompletionHandler() { + HttpClient.asyncHttpGet(url, null, params, new Callback() { @Override - public Integer onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { + public void onReceive(RestResult result) { + if (!result.ok()) { Loggers.RAFT - .error("[NACOS-RAFT] get peer failed: {}, peer: {}", response.getResponseBody(), - peer.ip); + .error("[NACOS-RAFT] get peer failed: {}, peer: {}", result.getCode(), peer.ip); peer.state = RaftPeer.State.FOLLOWER; - return 1; + return; } - update(JacksonUtils.toObj(response.getResponseBody(), RaftPeer.class)); - - return 0; + update(JacksonUtils.toObj(result.getData(), RaftPeer.class)); + } + + @Override + public void onError(Throwable throwable) { + + } + + @Override + public void onCancel() { + } }); } catch (Exception e) { @@ -317,7 +323,7 @@ public class RaftPeerSet extends MemberChangeListener implements Closeable { @Override public void onEvent(MembersChangeEvent event) { Collection members = event.getMembers(); - if (oldMembers == null) { + if (oldMembers.isEmpty()) { oldMembers = new HashSet<>(members); } else { oldMembers.removeAll(members); @@ -326,7 +332,7 @@ public class RaftPeerSet extends MemberChangeListener implements Closeable { if (!oldMembers.isEmpty()) { changePeers(members); } - + oldMembers.clear(); oldMembers.addAll(members); } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftProxy.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftProxy.java index 7fb10d6c1..6b7833b21 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftProxy.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftProxy.java @@ -16,13 +16,13 @@ package com.alibaba.nacos.naming.consistency.persistent.raft; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.naming.misc.HttpClient; import com.alibaba.nacos.naming.misc.UtilsAndCommons; import org.springframework.http.HttpMethod; import org.springframework.stereotype.Component; -import java.net.HttpURLConnection; import java.util.Map; /** @@ -50,9 +50,9 @@ public class RaftProxy { } String url = "http://" + server + ApplicationUtils.getContextPath() + api; - HttpClient.HttpResult result = HttpClient.httpGet(url, null, params); - if (result.code != HttpURLConnection.HTTP_OK) { - throw new IllegalStateException("leader failed, caused by: " + result.content); + RestResult result = HttpClient.httpGet(url, null, params); + if (!result.ok()) { + throw new IllegalStateException("leader failed, caused by: " + result.getMessage()); } } @@ -71,7 +71,7 @@ public class RaftProxy { server = server + UtilsAndCommons.IP_PORT_SPLITER + ApplicationUtils.getPort(); } String url = "http://" + server + ApplicationUtils.getContextPath() + api; - HttpClient.HttpResult result; + RestResult result; switch (method) { case GET: result = HttpClient.httpGet(url, null, params); @@ -86,8 +86,8 @@ public class RaftProxy { throw new RuntimeException("unsupported method:" + method); } - if (result.code != HttpURLConnection.HTTP_OK) { - throw new IllegalStateException("leader failed, caused by: " + result.content); + if (!result.ok()) { + throw new IllegalStateException("leader failed, caused by: " + result.getMessage()); } } @@ -108,9 +108,9 @@ public class RaftProxy { } String url = "http://" + server + ApplicationUtils.getContextPath() + api; - HttpClient.HttpResult result = HttpClient.httpPostLarge(url, headers, content); - if (result.code != HttpURLConnection.HTTP_OK) { - throw new IllegalStateException("leader failed, caused by: " + result.content); + RestResult result = HttpClient.httpPostLarge(url, headers, content); + if (!result.ok()) { + throw new IllegalStateException("leader failed, caused by: " + result.getMessage()); } } } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftStore.java b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftStore.java index d2b366d13..5a56c60ee 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftStore.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/consistency/persistent/raft/RaftStore.java @@ -24,8 +24,8 @@ import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.consistency.DataOperation; import com.alibaba.nacos.naming.consistency.Datum; import com.alibaba.nacos.naming.consistency.KeyBuilder; -import com.alibaba.nacos.naming.consistency.persistent.PersistentNotifier; import com.alibaba.nacos.naming.consistency.ValueChangeEvent; +import com.alibaba.nacos.naming.consistency.persistent.PersistentNotifier; import com.alibaba.nacos.naming.core.Instance; import com.alibaba.nacos.naming.core.Instances; import com.alibaba.nacos.naming.core.Service; @@ -49,14 +49,12 @@ import java.util.Map; import java.util.Properties; import static com.alibaba.nacos.naming.misc.UtilsAndCommons.DATA_BASE_DIR; -import static com.alibaba.nacos.naming.misc.UtilsAndCommons.RAFT_CACHE_FILE_PREFIX; -import static com.alibaba.nacos.naming.misc.UtilsAndCommons.RAFT_CACHE_FILE_SUFFIX; /** * Raft store. * - * @deprecated will remove in 1.4.x * @author nacos + * @deprecated will remove in 1.4.x */ @Deprecated @Component @@ -75,8 +73,7 @@ public class RaftStore implements Closeable { * @param datums cached datum map * @throws Exception any exception during load */ - public synchronized void loadDatums(PersistentNotifier notifier, Map datums) - throws Exception { + public synchronized void loadDatums(PersistentNotifier notifier, Map datums) throws Exception { Datum datum; long start = System.currentTimeMillis(); @@ -87,8 +84,8 @@ public class RaftStore implements Closeable { if (datum != null) { datums.put(datum.key, datum); if (notifier != null) { - NotifyCenter - .publishEvent(ValueChangeEvent.builder().key(datum.key).action(DataOperation.CHANGE).build()); + NotifyCenter.publishEvent( + ValueChangeEvent.builder().key(datum.key).action(DataOperation.CHANGE).build()); } } } @@ -137,8 +134,7 @@ public class RaftStore implements Closeable { Loggers.RAFT.warn("warning: encountered directory in cache dir: {}", cache.getAbsolutePath()); } - if (!StringUtils.equals(cache.getName(), encodeDatumKey(key)) && !StringUtils - .equals(cache.getName(), encodeDatumKey(key) + RAFT_CACHE_FILE_SUFFIX)) { + if (!StringUtils.equals(cache.getName(), encodeDatumKey(key))) { continue; } @@ -149,12 +145,8 @@ public class RaftStore implements Closeable { return null; } - private boolean isDatumCacheFile(String fileName) { - return fileName.endsWith(RAFT_CACHE_FILE_SUFFIX) || fileName.startsWith(RAFT_CACHE_FILE_PREFIX); - } - private synchronized Datum readDatum(File file, String namespaceId) throws IOException { - if (!isDatumCacheFile(file.getName())) { + if (!KeyBuilder.isDatumCacheFile(file.getName())) { return null; } ByteBuffer buffer; @@ -251,14 +243,6 @@ public class RaftStore implements Closeable { return fileName; } - private File cacheFile(String cacheFileName) { - File cacheFile = new File(cacheFileName); - if (cacheFile.exists()) { - return cacheFile; - } - return new File(cacheFileName + RAFT_CACHE_FILE_SUFFIX); - } - /** * Write datum to cache file. * @@ -269,7 +253,7 @@ public class RaftStore implements Closeable { String namespaceId = KeyBuilder.getNamespace(datum.key); - File cacheFile = cacheFile(cacheFileName(namespaceId, datum.key)); + File cacheFile = new File(cacheFileName(namespaceId, datum.key)); if (!cacheFile.exists() && !cacheFile.getParentFile().mkdirs() && !cacheFile.createNewFile()) { MetricsMonitor.getDiskException().increment(); @@ -301,7 +285,7 @@ public class RaftStore implements Closeable { String oldDatumKey = datum.key .replace(Constants.DEFAULT_GROUP + Constants.SERVICE_INFO_SPLITER, StringUtils.EMPTY); - cacheFile = cacheFile(cacheFileName(namespaceId, oldDatumKey)); + cacheFile = new File(cacheFileName(namespaceId, oldDatumKey)); if (cacheFile.exists() && !cacheFile.delete()) { Loggers.RAFT.error("[RAFT-DELETE] failed to delete old format datum: {}, value: {}", datum.key, datum.value); diff --git a/naming/src/main/java/com/alibaba/nacos/naming/controllers/ApiController.java b/naming/src/main/java/com/alibaba/nacos/naming/controllers/ApiController.java index bc0e915af..dfce95127 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/controllers/ApiController.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/controllers/ApiController.java @@ -88,8 +88,8 @@ public class ApiController extends InstanceController { } if (CollectionUtils.isEmpty(domSet)) { - result.put("doms", JacksonUtils.transferToJsonNode(new HashSet<>())); result.put("count", 0); + result.set("doms", JacksonUtils.transferToJsonNode(new HashSet<>())); return result; } @@ -99,8 +99,8 @@ public class ApiController extends InstanceController { } } - result.put("doms", JacksonUtils.transferToJsonNode(doms)); result.put("count", doms.size()); + result.set("doms", JacksonUtils.transferToJsonNode(doms)); return result; } @@ -116,8 +116,8 @@ public class ApiController extends InstanceController { count += doms.get(namespaceId).size(); } - result.put("doms", JacksonUtils.transferToJsonNode(doms)); result.put("count", count); + result.set("doms", JacksonUtils.transferToJsonNode(doms)); return result; } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/controllers/DistroController.java b/naming/src/main/java/com/alibaba/nacos/naming/controllers/DistroController.java index b1e89d92b..7052b71a1 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/controllers/DistroController.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/controllers/DistroController.java @@ -84,8 +84,7 @@ public class DistroController { .isDefaultInstanceEphemeral()) { serviceManager.createEmptyService(namespaceId, serviceName, true); } - DistroHttpData distroHttpData = new DistroHttpData(createDistroKey(entry.getKey()), null, - entry.getValue()); + DistroHttpData distroHttpData = new DistroHttpData(createDistroKey(entry.getKey()), entry.getValue()); distroProtocol.onReceive(distroHttpData); } } @@ -101,7 +100,7 @@ public class DistroController { */ @PutMapping("/checksum") public ResponseEntity syncChecksum(@RequestParam String source, @RequestBody Map dataMap) { - DistroHttpData distroHttpData = new DistroHttpData(createDistroKey(source), null, dataMap); + DistroHttpData distroHttpData = new DistroHttpData(createDistroKey(source), dataMap); distroProtocol.onVerify(distroHttpData); return ResponseEntity.ok("ok"); } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/controllers/HealthController.java b/naming/src/main/java/com/alibaba/nacos/naming/controllers/HealthController.java index 85957f5f4..8b0e39fac 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/controllers/HealthController.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/controllers/HealthController.java @@ -22,7 +22,7 @@ import com.alibaba.nacos.api.naming.pojo.healthcheck.AbstractHealthChecker; import com.alibaba.nacos.auth.annotation.Secured; import com.alibaba.nacos.auth.common.ActionTypes; import com.alibaba.nacos.common.utils.JacksonUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.core.utils.WebUtils; import com.alibaba.nacos.naming.core.Instance; import com.alibaba.nacos.naming.core.Service; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/controllers/InstanceController.java b/naming/src/main/java/com/alibaba/nacos/naming/controllers/InstanceController.java index 287cb13b2..5f1ae15a3 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/controllers/InstanceController.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/controllers/InstanceController.java @@ -38,6 +38,7 @@ import com.alibaba.nacos.naming.push.ClientInfo; import com.alibaba.nacos.naming.push.DataSource; import com.alibaba.nacos.naming.push.PushService; import com.alibaba.nacos.naming.web.CanDistro; +import com.alibaba.nacos.naming.web.DistroFilter; import com.alibaba.nacos.naming.web.NamingResourceParser; import com.fasterxml.jackson.databind.node.ArrayNode; import com.fasterxml.jackson.databind.node.ObjectNode; @@ -115,9 +116,10 @@ public class InstanceController { @Secured(parser = NamingResourceParser.class, action = ActionTypes.WRITE) public String register(HttpServletRequest request) throws Exception { - final String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); final String namespaceId = WebUtils .optional(request, CommonParams.NAMESPACE_ID, Constants.DEFAULT_NAMESPACE_ID); + final String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); + checkServiceNameFormat(serviceName); final Instance instance = parseInstance(request); @@ -139,6 +141,7 @@ public class InstanceController { Instance instance = getIpAddress(request); String namespaceId = WebUtils.optional(request, CommonParams.NAMESPACE_ID, Constants.DEFAULT_NAMESPACE_ID); String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); + checkServiceNameFormat(serviceName); Service service = serviceManager.getService(namespaceId, serviceName); if (service == null) { @@ -161,9 +164,10 @@ public class InstanceController { @PutMapping @Secured(parser = NamingResourceParser.class, action = ActionTypes.WRITE) public String update(HttpServletRequest request) throws Exception { - final String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); final String namespaceId = WebUtils .optional(request, CommonParams.NAMESPACE_ID, Constants.DEFAULT_NAMESPACE_ID); + final String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); + checkServiceNameFormat(serviceName); final Instance instance = parseInstance(request); String agent = WebUtils.getUserAgent(request); @@ -190,8 +194,9 @@ public class InstanceController { @PatchMapping @Secured(parser = NamingResourceParser.class, action = ActionTypes.WRITE) public String patch(HttpServletRequest request) throws Exception { - String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); String namespaceId = WebUtils.optional(request, CommonParams.NAMESPACE_ID, Constants.DEFAULT_NAMESPACE_ID); + String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); + checkServiceNameFormat(serviceName); String ip = WebUtils.required(request, "ip"); String port = WebUtils.required(request, "port"); String cluster = WebUtils.optional(request, CommonParams.CLUSTER_NAME, StringUtils.EMPTY); @@ -242,8 +247,9 @@ public class InstanceController { public ObjectNode list(HttpServletRequest request) throws Exception { String namespaceId = WebUtils.optional(request, CommonParams.NAMESPACE_ID, Constants.DEFAULT_NAMESPACE_ID); - String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); + checkServiceNameFormat(serviceName); + String agent = WebUtils.getUserAgent(request); String clusters = WebUtils.optional(request, "clusters", StringUtils.EMPTY); String clientIP = WebUtils.optional(request, "clientIP", StringUtils.EMPTY); @@ -274,6 +280,7 @@ public class InstanceController { String namespaceId = WebUtils.optional(request, CommonParams.NAMESPACE_ID, Constants.DEFAULT_NAMESPACE_ID); String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); + checkServiceNameFormat(serviceName); String cluster = WebUtils.optional(request, CommonParams.CLUSTER_NAME, UtilsAndCommons.DEFAULT_CLUSTER_NAME); String ip = WebUtils.required(request, "ip"); int port = Integer.parseInt(WebUtils.required(request, "port")); @@ -301,8 +308,8 @@ public class InstanceController { result.put("clusterName", cluster); result.put("weight", instance.getWeight()); result.put("healthy", instance.isHealthy()); - result.put("metadata", JacksonUtils.transferToJsonNode(instance.getMetadata())); result.put("instanceId", instance.getInstanceId()); + result.set("metadata", JacksonUtils.transferToJsonNode(instance.getMetadata())); return result; } } @@ -344,8 +351,9 @@ public class InstanceController { ip = clientBeat.getIp(); port = clientBeat.getPort(); } - String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); String namespaceId = WebUtils.optional(request, CommonParams.NAMESPACE_ID, Constants.DEFAULT_NAMESPACE_ID); + String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); + checkServiceNameFormat(serviceName); Loggers.SRV_LOG.debug("[CLIENT-BEAT] full arguments: beat: {}, serviceName: {}", clientBeat, serviceName); Instance instance = serviceManager.getInstance(namespaceId, serviceName, clusterName, ip, port); @@ -413,7 +421,7 @@ public class InstanceController { namespaceId = Constants.DEFAULT_NAMESPACE_ID; serviceName = key; } - + checkServiceNameFormat(serviceName); Service service = serviceManager.getService(namespaceId, serviceName); if (service == null) { @@ -433,6 +441,26 @@ public class InstanceController { return result; } + /** + * check combineServiceName format. the serviceName can't be blank. some relational logic in {@link + * DistroFilter#doFilter}, it will handle combineServiceName in some case, you should know it. + * 

+     * serviceName = "@@"; the length = 0; illegal
+     * serviceName = "group@@"; the length = 1; illegal
+     * serviceName = "@@serviceName"; the length = 2; legal
+     * serviceName = "group@@serviceName"; the length = 2; legal
+     *
+ * + * @param combineServiceName such as: groupName@@serviceName + */ + private void checkServiceNameFormat(String combineServiceName) { + String[] split = combineServiceName.split(Constants.SERVICE_INFO_SPLITER); + if (split.length <= 1) { + throw new IllegalArgumentException( + "Param 'serviceName' is illegal, it should be format as 'groupName@@serviceName"); + } + } + private Instance parseInstance(HttpServletRequest request) throws Exception { String serviceName = WebUtils.required(request, CommonParams.SERVICE_NAME); @@ -568,7 +596,6 @@ public class InstanceController { result.put("dom", NamingUtils.getServiceName(serviceName)); } - result.put("hosts", JacksonUtils.createEmptyArrayNode()); result.put("name", serviceName); result.put("cacheMillis", cacheMillis); result.put("lastRefTime", System.currentTimeMillis()); @@ -576,7 +603,8 @@ public class InstanceController { result.put("useSpecifiedURL", false); result.put("clusters", clusters); result.put("env", env); - result.put("metadata", JacksonUtils.transferToJsonNode(service.getMetadata())); + result.set("hosts", JacksonUtils.createEmptyArrayNode()); + result.set("metadata", JacksonUtils.transferToJsonNode(service.getMetadata())); return result; } @@ -637,7 +665,7 @@ public class InstanceController { ipObj.put("healthy", entry.getKey()); ipObj.put("marked", instance.isMarked()); ipObj.put("instanceId", instance.getInstanceId()); - ipObj.put("metadata", JacksonUtils.transferToJsonNode(instance.getMetadata())); + ipObj.set("metadata", JacksonUtils.transferToJsonNode(instance.getMetadata())); ipObj.put("enabled", instance.isEnabled()); ipObj.put("weight", instance.getWeight()); ipObj.put("clusterName", instance.getClusterName()); diff --git a/naming/src/main/java/com/alibaba/nacos/naming/controllers/OperatorController.java b/naming/src/main/java/com/alibaba/nacos/naming/controllers/OperatorController.java index f0500924c..180c93141 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/controllers/OperatorController.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/controllers/OperatorController.java @@ -23,7 +23,7 @@ import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.NodeState; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.cluster.ServerListManager; import com.alibaba.nacos.naming.cluster.ServerStatusManager; import com.alibaba.nacos.naming.consistency.persistent.raft.RaftCore; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/controllers/RaftController.java b/naming/src/main/java/com/alibaba/nacos/naming/controllers/RaftController.java index ebef44dd8..ea677e638 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/controllers/RaftController.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/controllers/RaftController.java @@ -17,6 +17,7 @@ package com.alibaba.nacos.naming.controllers; import com.alibaba.nacos.api.exception.NacosException; +import com.alibaba.nacos.common.utils.ConcurrentHashSet; import com.alibaba.nacos.common.utils.IoUtils; import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.core.utils.WebUtils; @@ -37,7 +38,6 @@ import com.fasterxml.jackson.core.type.TypeReference; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ArrayNode; import com.fasterxml.jackson.databind.node.ObjectNode; - import org.apache.commons.lang3.StringUtils; import org.springframework.web.bind.annotation.DeleteMapping; import org.springframework.web.bind.annotation.GetMapping; @@ -57,9 +57,9 @@ import java.util.Map; /** * Methods for Raft consistency protocol. These methods should only be invoked by Nacos server itself. * - * @deprecated will remove in 1.4.x * @author nkorange * @since 1.0.0 + * @deprecated will remove in 1.4.x */ @Deprecated @RestController @@ -72,7 +72,7 @@ public class RaftController { private final ServiceManager serviceManager; private final RaftCore raftCore; - + private final ClusterVersionJudgement versionJudgement; public RaftController(RaftConsistencyServiceImpl raftConsistencyService, ServiceManager serviceManager, @@ -387,7 +387,7 @@ public class RaftController { throw new IllegalStateException("old raft protocol already stop"); } ObjectNode result = JacksonUtils.createEmptyJsonNode(); - Map> listeners = raftCore.getListeners(); + Map> listeners = raftCore.getListeners(); ArrayNode listenerArray = JacksonUtils.createEmptyArrayNode(); for (String key : listeners.keySet()) { diff --git a/naming/src/main/java/com/alibaba/nacos/naming/core/DistroMapper.java b/naming/src/main/java/com/alibaba/nacos/naming/core/DistroMapper.java index b5ec07095..a4212b5e0 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/core/DistroMapper.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/core/DistroMapper.java @@ -22,7 +22,7 @@ import com.alibaba.nacos.core.cluster.MemberUtils; import com.alibaba.nacos.core.cluster.MembersChangeEvent; import com.alibaba.nacos.core.cluster.NodeState; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.misc.Loggers; import com.alibaba.nacos.naming.misc.SwitchDomain; import org.apache.commons.collections.CollectionUtils; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/core/Service.java b/naming/src/main/java/com/alibaba/nacos/naming/core/Service.java index c687c823f..5facf1709 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/core/Service.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/core/Service.java @@ -16,10 +16,10 @@ package com.alibaba.nacos.naming.core; -import com.alibaba.nacos.api.common.Constants; import com.alibaba.nacos.common.utils.JacksonUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.common.utils.MD5Utils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.api.common.Constants; import com.alibaba.nacos.naming.consistency.KeyBuilder; import com.alibaba.nacos.naming.consistency.RecordListener; import com.alibaba.nacos.naming.healthcheck.ClientBeatCheckTask; @@ -35,6 +35,7 @@ import com.alibaba.nacos.naming.selector.Selector; import com.fasterxml.jackson.annotation.JsonIgnore; import com.fasterxml.jackson.annotation.JsonInclude; import com.fasterxml.jackson.annotation.JsonInclude.Include; + import org.apache.commons.collections.CollectionUtils; import org.apache.commons.collections.ListUtils; import org.apache.commons.lang3.StringUtils; @@ -51,7 +52,8 @@ import java.util.Map; * Service of Nacos server side * *

We introduce a 'service --> cluster --> instance' model, in which service stores a list of clusters, which - * contain a list of instances. + * contain + * a list of instances. * *

his class inherits from Service in API module and stores some fields that do not have to expose to client. * @@ -62,8 +64,6 @@ public class Service extends com.alibaba.nacos.api.naming.pojo.Service implement private static final String SERVICE_NAME_SYNTAX = "[0-9a-zA-Z@\\.:_-]+"; - private static final long serialVersionUID = -1341508935415925039L; - @JsonIgnore private ClientBeatCheckTask clientBeatCheckTask = new ClientBeatCheckTask(this); diff --git a/naming/src/main/java/com/alibaba/nacos/naming/core/ServiceManager.java b/naming/src/main/java/com/alibaba/nacos/naming/core/ServiceManager.java index 47365be17..5860b5884 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/core/ServiceManager.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/core/ServiceManager.java @@ -40,7 +40,14 @@ import com.alibaba.nacos.naming.push.PushService; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ArrayNode; import com.google.common.collect.Sets; +import org.apache.commons.lang3.ArrayUtils; +import org.apache.commons.lang3.StringUtils; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.stereotype.Component; +import org.springframework.util.CollectionUtils; +import javax.annotation.PostConstruct; +import javax.annotation.Resource; import java.util.ArrayList; import java.util.Collection; import java.util.HashMap; @@ -56,14 +63,6 @@ import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; import java.util.stream.Collectors; import java.util.stream.Stream; -import javax.annotation.PostConstruct; -import javax.annotation.Resource; - -import org.apache.commons.lang3.ArrayUtils; -import org.apache.commons.lang3.StringUtils; -import org.springframework.beans.factory.annotation.Value; -import org.springframework.stereotype.Component; -import org.springframework.util.CollectionUtils; /** * Core manager storing all services in Nacos. @@ -229,10 +228,14 @@ public class ServiceManager implements RecordListener { if (service != null) { service.destroy(); - consistencyService.remove(KeyBuilder.buildInstanceListKey(namespace, name, true)); - - consistencyService.remove(KeyBuilder.buildInstanceListKey(namespace, name, false)); - + String ephemeralInstanceListKey = KeyBuilder.buildInstanceListKey(namespace, name, true); + String persistInstanceListKey = KeyBuilder.buildInstanceListKey(namespace, name, false); + consistencyService.remove(ephemeralInstanceListKey); + consistencyService.remove(persistInstanceListKey); + + // remove listeners of key to avoid mem leak + consistencyService.unListen(ephemeralInstanceListKey, service); + consistencyService.unListen(persistInstanceListKey, service); consistencyService.unListen(KeyBuilder.buildServiceMetaKey(namespace, name), service); Loggers.SRV_LOG.info("[DEAD-SERVICE] {}", service.toJson()); } @@ -631,7 +634,7 @@ public class ServiceManager implements RecordListener { } Map instanceMap; - if (datum != null) { + if (datum != null && null != datum.value) { instanceMap = setValid(((Instances) datum.value).getInstanceList(), currentInstances); } else { instanceMap = new HashMap<>(ips.length); diff --git a/naming/src/main/java/com/alibaba/nacos/naming/core/SubscribeManager.java b/naming/src/main/java/com/alibaba/nacos/naming/core/SubscribeManager.java index 1cd1651c1..fcefcc0b3 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/core/SubscribeManager.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/core/SubscribeManager.java @@ -17,10 +17,11 @@ package com.alibaba.nacos.naming.core; import com.alibaba.nacos.api.naming.CommonParams; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.misc.HttpClient; import com.alibaba.nacos.naming.misc.NetUtils; import com.alibaba.nacos.naming.misc.UtilsAndCommons; @@ -31,7 +32,6 @@ import org.apache.commons.collections.CollectionUtils; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; -import java.net.HttpURLConnection; import java.util.ArrayList; import java.util.Collections; import java.util.HashMap; @@ -97,13 +97,13 @@ public class SubscribeManager { continue; } - HttpClient.HttpResult result = HttpClient.httpGet( + RestResult result = HttpClient.httpGet( "http://" + server.getAddress() + ApplicationUtils.getContextPath() + UtilsAndCommons.NACOS_NAMING_CONTEXT + SUBSCRIBER_ON_SYNC_URL, new ArrayList<>(), paramValues); - if (HttpURLConnection.HTTP_OK == result.code) { - Subscribers subscribers = JacksonUtils.toObj(result.content, Subscribers.class); + if (result.ok()) { + Subscribers subscribers = JacksonUtils.toObj(result.getData(), Subscribers.class); subscriberList.addAll(subscribers.getSubscribers()); } } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/ClientBeatCheckTask.java b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/ClientBeatCheckTask.java index 3860d002f..6de0fd0db 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/ClientBeatCheckTask.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/ClientBeatCheckTask.java @@ -16,8 +16,10 @@ package com.alibaba.nacos.naming.healthcheck; +import com.alibaba.nacos.common.http.Callback; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.utils.JacksonUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.consistency.KeyBuilder; import com.alibaba.nacos.naming.core.DistroMapper; import com.alibaba.nacos.naming.core.Instance; @@ -31,11 +33,7 @@ import com.alibaba.nacos.naming.misc.SwitchDomain; import com.alibaba.nacos.naming.misc.UtilsAndCommons; import com.alibaba.nacos.naming.push.PushService; import com.fasterxml.jackson.annotation.JsonIgnore; -import com.ning.http.client.AsyncCompletionHandler; -import com.ning.http.client.Response; -import java.io.Serializable; -import java.net.HttpURLConnection; import java.util.List; /** @@ -43,9 +41,7 @@ import java.util.List; * * @author nkorange */ -public class ClientBeatCheckTask implements Runnable, Serializable { - - private static final long serialVersionUID = 1518045401492647614L; +public class ClientBeatCheckTask implements Runnable { private Service service; @@ -143,15 +139,26 @@ public class ClientBeatCheckTask implements Runnable, Serializable { + UtilsAndCommons.NACOS_NAMING_CONTEXT + "/instance?" + request.toUrl(); // delete instance asynchronously: - HttpClient.asyncHttpDelete(url, null, null, new AsyncCompletionHandler() { + HttpClient.asyncHttpDelete(url, null, null, new Callback() { @Override - public Object onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { + public void onReceive(RestResult result) { + if (!result.ok()) { Loggers.SRV_LOG .error("[IP-DEAD] failed to delete ip automatically, ip: {}, caused {}, resp code: {}", - instance.toJson(), response.getResponseBody(), response.getStatusCode()); + instance.toJson(), result.getMessage(), result.getCode()); } - return null; + } + + @Override + public void onError(Throwable throwable) { + Loggers.SRV_LOG + .error("[IP-DEAD] failed to delete ip automatically, ip: {}, error: {}", instance.toJson(), + throwable); + } + + @Override + public void onCancel() { + } }); diff --git a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/ClientBeatProcessor.java b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/ClientBeatProcessor.java index 213c27198..a2b147b6c 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/ClientBeatProcessor.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/ClientBeatProcessor.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.naming.healthcheck; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.core.Cluster; import com.alibaba.nacos.naming.core.Instance; import com.alibaba.nacos.naming.core.Service; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckCommon.java b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckCommon.java index 41a345fc9..ac224b5d0 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckCommon.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckCommon.java @@ -16,10 +16,11 @@ package com.alibaba.nacos.naming.healthcheck; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.core.Cluster; import com.alibaba.nacos.naming.core.DistroMapper; import com.alibaba.nacos.naming.core.Instance; @@ -34,7 +35,6 @@ import com.alibaba.nacos.naming.push.PushService; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; -import java.net.HttpURLConnection; import java.util.Arrays; import java.util.Collection; import java.util.HashMap; @@ -92,11 +92,11 @@ public class HealthCheckCommon { JacksonUtils.toJson(list)); } - HttpClient.HttpResult httpResult = HttpClient.httpPost( + RestResult httpResult = HttpClient.httpPost( "http://" + server.getAddress() + ApplicationUtils.getContextPath() + UtilsAndCommons.NACOS_NAMING_CONTEXT + "/api/healthCheckResult", null, params); - if (httpResult.code != HttpURLConnection.HTTP_OK) { + if (!httpResult.ok()) { Loggers.EVT_LOG.warn("[HEALTH-CHECK-SYNC] failed to send result to {}, result: {}", server, JacksonUtils.toJson(list)); } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckReactor.java b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckReactor.java index a095fe595..cfc5054a7 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckReactor.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckReactor.java @@ -66,6 +66,7 @@ public class HealthCheckReactor { } try { scheduledFuture.cancel(true); + futureMap.remove(task.taskKey()); } catch (Exception e) { Loggers.EVT_LOG.error("[CANCEL-CHECK] cancel failed!", e); } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckTask.java b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckTask.java index 511cc683f..8599fb8ce 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckTask.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/healthcheck/HealthCheckTask.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.naming.healthcheck; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.core.Cluster; import com.alibaba.nacos.naming.core.DistroMapper; import com.alibaba.nacos.naming.misc.Loggers; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/misc/HttpClient.java b/naming/src/main/java/com/alibaba/nacos/naming/misc/HttpClient.java index 7ce2daf3e..07941e9dd 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/misc/HttpClient.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/misc/HttpClient.java @@ -17,49 +17,24 @@ package com.alibaba.nacos.naming.misc; import com.alibaba.nacos.common.constant.HttpHeaderConsts; +import com.alibaba.nacos.common.http.Callback; +import com.alibaba.nacos.common.http.HttpClientConfig; +import com.alibaba.nacos.common.http.client.NacosAsyncRestTemplate; +import com.alibaba.nacos.common.http.client.NacosRestTemplate; +import com.alibaba.nacos.common.http.param.Header; +import com.alibaba.nacos.common.http.param.MediaType; +import com.alibaba.nacos.common.http.param.Query; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.utils.HttpMethod; -import com.alibaba.nacos.common.utils.IoUtils; import com.alibaba.nacos.common.utils.VersionUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.ning.http.client.AsyncCompletionHandler; -import com.ning.http.client.AsyncHttpClient; -import com.ning.http.client.AsyncHttpClientConfig; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.collections.CollectionUtils; import org.apache.commons.collections.MapUtils; import org.apache.commons.lang3.StringUtils; -import org.apache.http.HeaderElement; -import org.apache.http.HttpEntity; -import org.apache.http.HttpHeaders; -import org.apache.http.HttpResponse; -import org.apache.http.NameValuePair; -import org.apache.http.client.config.RequestConfig; -import org.apache.http.client.entity.UrlEncodedFormEntity; -import org.apache.http.client.methods.HttpEntityEnclosingRequestBase; -import org.apache.http.client.methods.HttpPost; -import org.apache.http.client.methods.HttpPut; -import org.apache.http.entity.ByteArrayEntity; -import org.apache.http.entity.ContentType; -import org.apache.http.entity.StringEntity; -import org.apache.http.impl.client.CloseableHttpClient; -import org.apache.http.impl.client.HttpClientBuilder; -import org.apache.http.impl.client.HttpClients; -import org.apache.http.message.BasicNameValuePair; -import java.io.IOException; -import java.io.InputStream; -import java.io.UnsupportedEncodingException; -import java.net.HttpURLConnection; -import java.net.URI; -import java.net.URL; -import java.net.URLEncoder; -import java.util.ArrayList; -import java.util.Collections; import java.util.HashMap; -import java.util.Iterator; import java.util.List; import java.util.Map; -import java.util.concurrent.TimeUnit; -import java.util.zip.GZIPInputStream; /** * Http Client. @@ -72,34 +47,11 @@ public class HttpClient { private static final int CON_TIME_OUT_MILLIS = 5000; - private static AsyncHttpClient asyncHttpClient; + private static final NacosRestTemplate SYNC_NACOS_REST_TEMPLATE = HttpClientManager.getNacosRestTemplate(); - private static CloseableHttpClient postClient; + private static final NacosRestTemplate APACHE_SYNC_NACOS_REST_TEMPLATE = HttpClientManager.getApacheRestTemplate(); - static { - AsyncHttpClientConfig.Builder builder = new AsyncHttpClientConfig.Builder(); - builder.setMaximumConnectionsTotal(-1); - builder.setMaximumConnectionsPerHost(128); - builder.setAllowPoolingConnection(true); - builder.setFollowRedirects(false); - builder.setIdleConnectionTimeoutInMs(TIME_OUT_MILLIS); - builder.setConnectionTimeoutInMs(CON_TIME_OUT_MILLIS); - builder.setCompressionEnabled(true); - builder.setIOThreadMultiplier(1); - builder.setMaxRequestRetry(0); - builder.setUserAgent(UtilsAndCommons.SERVER_VERSION); - - asyncHttpClient = new AsyncHttpClient(builder.build()); - - HttpClientBuilder builder2 = HttpClients.custom(); - builder2.setUserAgent(UtilsAndCommons.SERVER_VERSION); - builder2.setConnectionTimeToLive(CON_TIME_OUT_MILLIS, TimeUnit.MILLISECONDS); - builder2.setMaxConnPerRoute(-1); - builder2.setMaxConnTotal(-1); - builder2.disableAutomaticRetries(); - - postClient = builder2.build(); - } + private static final NacosAsyncRestTemplate ASYNC_REST_TEMPLATE = HttpClientManager.getAsyncRestTemplate(); /** * Request http delete method. @@ -107,11 +59,11 @@ public class HttpClient { * @param url url * @param headers headers * @param paramValues params - * @return {@link HttpResult} as response + * @return {@link RestResult} as response */ - public static HttpResult httpDelete(String url, List headers, Map paramValues) { + public static RestResult httpDelete(String url, List headers, Map paramValues) { return request(url, headers, paramValues, StringUtils.EMPTY, CON_TIME_OUT_MILLIS, TIME_OUT_MILLIS, "UTF-8", - "DELETE"); + HttpMethod.DELETE); } /** @@ -120,11 +72,11 @@ public class HttpClient { * @param url url * @param headers headers * @param paramValues params - * @return {@link HttpResult} as response + * @return {@link RestResult} as response */ - public static HttpResult httpGet(String url, List headers, Map paramValues) { + public static RestResult httpGet(String url, List headers, Map paramValues) { return request(url, headers, paramValues, StringUtils.EMPTY, CON_TIME_OUT_MILLIS, TIME_OUT_MILLIS, "UTF-8", - "GET"); + HttpMethod.GET); } /** @@ -138,39 +90,31 @@ public class HttpClient { * @param readTimeout timeout of request * @param encoding charset of request * @param method http method - * @return {@link HttpResult} as response + * @return {@link RestResult} as response */ - public static HttpResult request(String url, List headers, Map paramValues, String body, - int connectTimeout, int readTimeout, String encoding, String method) { - HttpURLConnection conn = null; + public static RestResult request(String url, List headers, Map paramValues, + String body, int connectTimeout, int readTimeout, String encoding, String method) { + Header header = Header.newInstance(); + if (CollectionUtils.isNotEmpty(headers)) { + header.addAll(headers); + } + header.setContentType(MediaType.APPLICATION_FORM_URLENCODED); + header.addParam(HttpHeaderConsts.CLIENT_VERSION_HEADER, VersionUtils.version); + header.addParam(HttpHeaderConsts.USER_AGENT_HEADER, UtilsAndCommons.SERVER_VERSION); + header.addParam(HttpHeaderConsts.REQUEST_SOURCE_HEADER, ApplicationUtils.getLocalAddress()); + header.addParam(HttpHeaderConsts.ACCEPT_CHARSET, encoding); + + HttpClientConfig httpClientConfig = HttpClientConfig.builder().setConTimeOutMillis(connectTimeout) + .setReadTimeOutMillis(readTimeout).build(); + Query query = Query.newInstance().initParams(paramValues); + query.addParam("encoding", "UTF-8"); + query.addParam("nofix", "1"); try { - String encodedContent = encodingParams(paramValues, encoding); - url += StringUtils.isBlank(encodedContent) ? StringUtils.EMPTY : ("?" + encodedContent); - - conn = (HttpURLConnection) new URL(url).openConnection(); - conn.setConnectTimeout(connectTimeout); - conn.setReadTimeout(readTimeout); - conn.setRequestMethod(method); - - setHeaders(conn, headers, encoding); - - if (StringUtils.isNotBlank(body)) { - conn.setDoOutput(true); - byte[] b = body.getBytes(); - conn.setRequestProperty("Content-Length", String.valueOf(b.length)); - conn.getOutputStream().write(b, 0, b.length); - conn.getOutputStream().flush(); - conn.getOutputStream().close(); - } - - conn.connect(); - - return getResult(conn); + return SYNC_NACOS_REST_TEMPLATE + .exchange(url, httpClientConfig, header, query, body, method, String.class); } catch (Exception e) { Loggers.SRV_LOG.warn("Exception while request: {}, caused: {}", url, e); - return new HttpResult(500, e.toString(), Collections.emptyMap()); - } finally { - IoUtils.closeQuietly(conn); + return RestResult.builder().withCode(500).withMsg(e.toString()).build(); } } @@ -180,11 +124,11 @@ public class HttpClient { * @param url url * @param headers headers * @param paramValues params - * @param handler callback after request execute + * @param callback callback after request execute */ public static void asyncHttpGet(String url, List headers, Map paramValues, - AsyncCompletionHandler handler) throws Exception { - asyncHttpRequest(url, headers, paramValues, handler, HttpMethod.GET); + Callback callback) throws Exception { + asyncHttpRequest(url, headers, paramValues, callback, HttpMethod.GET); } /** @@ -193,11 +137,11 @@ public class HttpClient { * @param url url * @param headers headers * @param paramValues params - * @param handler callback after request execute + * @param callback callback after request execute */ public static void asyncHttpPost(String url, List headers, Map paramValues, - AsyncCompletionHandler handler) throws Exception { - asyncHttpRequest(url, headers, paramValues, handler, HttpMethod.POST); + Callback callback) throws Exception { + asyncHttpRequest(url, headers, paramValues, callback, HttpMethod.POST); } /** @@ -206,11 +150,11 @@ public class HttpClient { * @param url url * @param headers headers * @param paramValues params - * @param handler callback after request execute + * @param callback callback after request execute */ public static void asyncHttpDelete(String url, List headers, Map paramValues, - AsyncCompletionHandler handler) throws Exception { - asyncHttpRequest(url, headers, paramValues, handler, HttpMethod.DELETE); + Callback callback) throws Exception { + asyncHttpRequest(url, headers, paramValues, callback, HttpMethod.DELETE); } /** @@ -223,124 +167,84 @@ public class HttpClient { * @throws Exception exception when request */ public static void asyncHttpRequest(String url, List headers, Map paramValues, - AsyncCompletionHandler handler, String method) throws Exception { - if (!MapUtils.isEmpty(paramValues)) { - String encodedContent = encodingParams(paramValues, "UTF-8"); - url += (null == encodedContent) ? "" : ("?" + encodedContent); - } + Callback callback, String method) throws Exception { - AsyncHttpClient.BoundRequestBuilder builder; + Query query = Query.newInstance().initParams(paramValues); + query.addParam("encoding", "UTF-8"); + query.addParam("nofix", "1"); + + Header header = Header.newInstance(); + if (CollectionUtils.isNotEmpty(headers)) { + header.addAll(headers); + } + header.addParam(HttpHeaderConsts.ACCEPT_CHARSET, "UTF-8"); switch (method) { case HttpMethod.GET: - builder = asyncHttpClient.prepareGet(url); + ASYNC_REST_TEMPLATE.get(url, header, query, String.class, callback); break; case HttpMethod.POST: - builder = asyncHttpClient.preparePost(url); + ASYNC_REST_TEMPLATE.postForm(url, header, paramValues, String.class, callback); break; case HttpMethod.PUT: - builder = asyncHttpClient.preparePut(url); + ASYNC_REST_TEMPLATE.putForm(url, header, paramValues, String.class, callback); break; case HttpMethod.DELETE: - builder = asyncHttpClient.prepareDelete(url); + ASYNC_REST_TEMPLATE.delete(url, header, query, String.class, callback); break; default: throw new RuntimeException("not supported method:" + method); } - - if (!CollectionUtils.isEmpty(headers)) { - for (String header : headers) { - builder.setHeader(header.split("=")[0], header.split("=")[1]); - } - } - - builder.setHeader("Accept-Charset", "UTF-8"); - - if (handler != null) { - builder.execute(handler); - } else { - builder.execute(); - } } /** * Request http post method by async with large body. * - * @param url url - * @param headers headers - * @param content full request content - * @param handler callback after request execute + * @param url url + * @param headers headers + * @param content full request content + * @param callback callback after request execute */ - public static void asyncHttpPostLarge(String url, List headers, String content, - AsyncCompletionHandler handler) throws Exception { - asyncHttpPostLarge(url, headers, content.getBytes(), handler); + public static void asyncHttpPostLarge(String url, List headers, String content, Callback callback) + throws Exception { + asyncHttpPostLarge(url, headers, content.getBytes(), callback); } /** * Request http post method by async with large body. * - * @param url url - * @param headers headers - * @param content full request content - * @param handler callback after request execute + * @param url url + * @param headers headers + * @param content full request content + * @param callback callback after request execute */ - public static void asyncHttpPostLarge(String url, List headers, byte[] content, - AsyncCompletionHandler handler) throws Exception { - AsyncHttpClient.BoundRequestBuilder builder = asyncHttpClient.preparePost(url); - - if (!CollectionUtils.isEmpty(headers)) { - for (String header : headers) { - builder.setHeader(header.split("=")[0], header.split("=")[1]); - } - } - - builder.setBody(content); - - builder.setHeader("Content-Type", "application/json; charset=UTF-8"); - builder.setHeader("Accept-Charset", "UTF-8"); - builder.setHeader("Accept-Encoding", "gzip"); - builder.setHeader("Content-Encoding", "gzip"); - - if (handler != null) { - builder.execute(handler); - } else { - builder.execute(); + public static void asyncHttpPostLarge(String url, List headers, byte[] content, Callback callback) + throws Exception { + Header header = Header.newInstance(); + if (CollectionUtils.isNotEmpty(headers)) { + header.addAll(headers); } + ASYNC_REST_TEMPLATE.post(url, header, Query.EMPTY, content, String.class, callback); } /** * Request http delete method by async with large body. * - * @param url url - * @param headers headers - * @param content full request content - * @param handler callback after request execute + * @param url url + * @param headers headers + * @param content full request content + * @param callback callback after request execute */ - public static void asyncHttpDeleteLarge(String url, List headers, String content, - AsyncCompletionHandler handler) throws Exception { - AsyncHttpClient.BoundRequestBuilder builder = asyncHttpClient.prepareDelete(url); - - if (!CollectionUtils.isEmpty(headers)) { - for (String header : headers) { - builder.setHeader(header.split("=")[0], header.split("=")[1]); - } - } - - builder.setBody(content.getBytes()); - - builder.setHeader("Content-Type", "application/json; charset=UTF-8"); - builder.setHeader("Accept-Charset", "UTF-8"); - builder.setHeader("Accept-Encoding", "gzip"); - builder.setHeader("Content-Encoding", "gzip"); - - if (handler != null) { - builder.execute(handler); - } else { - builder.execute(); + public static void asyncHttpDeleteLarge(String url, List headers, String content, Callback callback) + throws Exception { + Header header = Header.newInstance(); + if (CollectionUtils.isNotEmpty(headers)) { + header.addAll(headers); } + ASYNC_REST_TEMPLATE.delete(url, header, content, String.class, callback); } - public static HttpResult httpPost(String url, List headers, Map paramValues) { + public static RestResult httpPost(String url, List headers, Map paramValues) { return httpPost(url, headers, paramValues, "UTF-8"); } @@ -351,109 +255,40 @@ public class HttpClient { * @param headers headers * @param paramValues params * @param encoding charset - * @return {@link HttpResult} as response + * @return {@link RestResult} as response */ - public static HttpResult httpPost(String url, List headers, Map paramValues, + public static RestResult httpPost(String url, List headers, Map paramValues, String encoding) { try { - - HttpPost httpost = new HttpPost(url); - - RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(5000) - .setConnectTimeout(5000).setSocketTimeout(5000).setRedirectsEnabled(true).setMaxRedirects(5) - .build(); - httpost.setConfig(requestConfig); - - List nvps = new ArrayList(); - - for (Map.Entry entry : paramValues.entrySet()) { - nvps.add(new BasicNameValuePair(entry.getKey(), entry.getValue())); + Header header = Header.newInstance(); + if (CollectionUtils.isNotEmpty(headers)) { + header.addAll(headers); } + header.addParam(HttpHeaderConsts.ACCEPT_CHARSET, encoding); - httpost.setEntity(new UrlEncodedFormEntity(nvps, encoding)); - HttpResponse response = postClient.execute(httpost); - HttpEntity entity = response.getEntity(); - - String charset = encoding; - if (entity.getContentType() != null) { - - HeaderElement[] headerElements = entity.getContentType().getElements(); - - if (headerElements != null && headerElements.length > 0 && headerElements[0] != null - && headerElements[0].getParameterByName("charset") != null) { - charset = headerElements[0].getParameterByName("charset").getValue(); - } - } - - return new HttpResult(response.getStatusLine().getStatusCode(), - IoUtils.toString(entity.getContent(), charset), Collections.emptyMap()); + HttpClientConfig httpClientConfig = HttpClientConfig.builder().setConTimeOutMillis(5000).setReadTimeOutMillis(5000) + .setConnectionRequestTimeout(5000).setMaxRedirects(5).build(); + return APACHE_SYNC_NACOS_REST_TEMPLATE.postForm(url, httpClientConfig, header, paramValues, String.class); } catch (Throwable e) { - return new HttpResult(500, e.toString(), Collections.emptyMap()); + return RestResult.builder().withCode(500).withMsg(e.toString()).build(); } } /** * Request http put method by async with large body. * - * @param url url - * @param headers headers - * @param content full request content - * @param handler callback after request execute + * @param url url + * @param headers headers + * @param content full request content + * @param callback callback after request execute */ public static void asyncHttpPutLarge(String url, Map headers, byte[] content, - AsyncCompletionHandler handler) throws Exception { - AsyncHttpClient.BoundRequestBuilder builder = asyncHttpClient.preparePut(url); - - if (!headers.isEmpty()) { - for (String headerKey : headers.keySet()) { - builder.setHeader(headerKey, headers.get(headerKey)); - } - } - - builder.setBody(content); - - builder.setHeader("Content-Type", "application/json; charset=UTF-8"); - builder.setHeader("Accept-Charset", "UTF-8"); - builder.setHeader("Accept-Encoding", "gzip"); - builder.setHeader("Content-Encoding", "gzip"); - - if (handler != null) { - builder.execute(handler); - } else { - builder.execute(); - } - } - - /** - * Request http get method by async with large body. - * - * @param url url - * @param headers headers - * @param content full request content - * @param handler callback after request execute - */ - public static void asyncHttpGetLarge(String url, Map headers, byte[] content, - AsyncCompletionHandler handler) throws Exception { - AsyncHttpClient.BoundRequestBuilder builder = asyncHttpClient.prepareGet(url); - - if (!headers.isEmpty()) { - for (String headerKey : headers.keySet()) { - builder.setHeader(headerKey, headers.get(headerKey)); - } - } - - builder.setBody(content); - - builder.setHeader("Content-Type", "application/json; charset=UTF-8"); - builder.setHeader("Accept-Charset", "UTF-8"); - builder.setHeader("Accept-Encoding", "gzip"); - builder.setHeader("Content-Encoding", "gzip"); - - if (handler != null) { - builder.execute(handler); - } else { - builder.execute(); + Callback callback) throws Exception { + Header header = Header.newInstance(); + if (MapUtils.isNotEmpty(headers)) { + header.addAll(headers); } + ASYNC_REST_TEMPLATE.put(url, header, Query.EMPTY, content, String.class, callback); } /** @@ -462,29 +297,17 @@ public class HttpClient { * @param url url * @param headers headers * @param content full request content - * @return {@link HttpResult} as response + * @return {@link RestResult} as response */ - public static HttpResult httpPutLarge(String url, Map headers, byte[] content) { - HttpClientBuilder builder = HttpClients.custom().setUserAgent(UtilsAndCommons.SERVER_VERSION) - .setConnectionTimeToLive(500, TimeUnit.MILLISECONDS); - try (CloseableHttpClient httpClient = builder.build();) { - - HttpPut httpPut = new HttpPut(url); - for (Map.Entry entry : headers.entrySet()) { - httpPut.setHeader(entry.getKey(), entry.getValue()); - } - httpPut.setEntity(new ByteArrayEntity(content, ContentType.APPLICATION_JSON)); - - HttpResponse response = httpClient.execute(httpPut); - HttpEntity entity = response.getEntity(); - - HeaderElement[] headerElements = entity.getContentType().getElements(); - String charset = headerElements[0].getParameterByName("charset").getValue(); - - return new HttpResult(response.getStatusLine().getStatusCode(), - IoUtils.toString(entity.getContent(), charset), Collections.emptyMap()); + public static RestResult httpPutLarge(String url, Map headers, byte[] content) { + Header header = Header.newInstance(); + if (MapUtils.isNotEmpty(headers)) { + header.addAll(headers); + } + try { + return APACHE_SYNC_NACOS_REST_TEMPLATE.put(url, header, Query.EMPTY, content, String.class); } catch (Exception e) { - return new HttpResult(500, e.toString(), Collections.emptyMap()); + return RestResult.builder().withCode(500).withMsg(e.toString()).build(); } } @@ -494,33 +317,17 @@ public class HttpClient { * @param url url * @param headers headers * @param content full request content - * @return {@link HttpResult} as response + * @return {@link RestResult} as response */ - public static HttpResult httpGetLarge(String url, Map headers, String content) { - HttpClientBuilder builder = HttpClients.custom(); - builder.setUserAgent(UtilsAndCommons.SERVER_VERSION); - builder.setConnectionTimeToLive(500, TimeUnit.MILLISECONDS); - try (CloseableHttpClient httpClient = builder.build();) { - - HttpGetWithEntity httpGetWithEntity = new HttpGetWithEntity(); - httpGetWithEntity.setURI(new URI(url)); - - for (Map.Entry entry : headers.entrySet()) { - httpGetWithEntity.setHeader(entry.getKey(), entry.getValue()); - } - - httpGetWithEntity.setEntity(new StringEntity(content, ContentType.create("application/json", "UTF-8"))); - - HttpResponse response = httpClient.execute(httpGetWithEntity); - HttpEntity entity = response.getEntity(); - - HeaderElement[] headerElements = entity.getContentType().getElements(); - String charset = headerElements[0].getParameterByName("charset").getValue(); - - return new HttpResult(response.getStatusLine().getStatusCode(), - IoUtils.toString(entity.getContent(), charset), Collections.emptyMap()); + public static RestResult httpGetLarge(String url, Map headers, String content) { + Header header = Header.newInstance(); + if (MapUtils.isNotEmpty(headers)) { + header.addAll(headers); + } + try { + return APACHE_SYNC_NACOS_REST_TEMPLATE.getLarge(url, header, Query.EMPTY, content, String.class); } catch (Exception e) { - return new HttpResult(500, e.toString(), Collections.emptyMap()); + return RestResult.builder().withCode(500).withMsg(e.toString()).build(); } } @@ -530,126 +337,20 @@ public class HttpClient { * @param url url * @param headers headers * @param content full request content - * @return {@link HttpResult} as response + * @return {@link RestResult} as response */ - public static HttpResult httpPostLarge(String url, Map headers, String content) { - HttpClientBuilder builder = HttpClients.custom(); - builder.setUserAgent(UtilsAndCommons.SERVER_VERSION); - builder.setConnectionTimeToLive(500, TimeUnit.MILLISECONDS); - try (CloseableHttpClient httpClient = builder.build();) { - - HttpPost httpost = new HttpPost(url); - - for (Map.Entry entry : headers.entrySet()) { - httpost.setHeader(entry.getKey(), entry.getValue()); - } - - httpost.setEntity(new StringEntity(content, ContentType.create("application/json", "UTF-8"))); - HttpResponse response = httpClient.execute(httpost); - HttpEntity entity = response.getEntity(); - - HeaderElement[] headerElements = entity.getContentType().getElements(); - String charset = headerElements[0].getParameterByName("charset").getValue(); - - return new HttpResult(response.getStatusLine().getStatusCode(), - IoUtils.toString(entity.getContent(), charset), Collections.emptyMap()); + public static RestResult httpPostLarge(String url, Map headers, String content) { + Header header = Header.newInstance(); + if (MapUtils.isNotEmpty(headers)) { + header.addAll(headers); + } + try { + return APACHE_SYNC_NACOS_REST_TEMPLATE.postJson(url, header, content, String.class); } catch (Exception e) { - return new HttpResult(500, e.toString(), Collections.emptyMap()); + return RestResult.builder().withCode(500).withMsg(e.toString()).build(); } } - private static HttpResult getResult(HttpURLConnection conn) throws IOException { - int respCode = conn.getResponseCode(); - - InputStream inputStream; - if (HttpURLConnection.HTTP_OK == respCode) { - inputStream = conn.getInputStream(); - } else { - inputStream = conn.getErrorStream(); - } - - Map respHeaders = new HashMap(conn.getHeaderFields().size()); - for (Map.Entry> entry : conn.getHeaderFields().entrySet()) { - respHeaders.put(entry.getKey(), entry.getValue().get(0)); - } - - String gzipEncoding = "gzip"; - - if (gzipEncoding.equals(respHeaders.get(HttpHeaders.CONTENT_ENCODING))) { - inputStream = new GZIPInputStream(inputStream); - } - - return new HttpResult(respCode, IoUtils.toString(inputStream, getCharset(conn)), respHeaders); - } - - private static String getCharset(HttpURLConnection conn) { - String contentType = conn.getContentType(); - if (StringUtils.isEmpty(contentType)) { - return "UTF-8"; - } - - String[] values = contentType.split(";"); - if (values.length == 0) { - return "UTF-8"; - } - - String charset = "UTF-8"; - for (String value : values) { - value = value.trim(); - - if (value.toLowerCase().startsWith("charset=")) { - charset = value.substring("charset=".length()); - } - } - - return charset; - } - - private static void setHeaders(HttpURLConnection conn, List headers, String encoding) { - if (null != headers) { - for (Iterator iter = headers.iterator(); iter.hasNext(); ) { - conn.addRequestProperty(iter.next(), iter.next()); - } - } - - conn.addRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + encoding); - conn.addRequestProperty("Accept-Charset", encoding); - conn.addRequestProperty(HttpHeaderConsts.CLIENT_VERSION_HEADER, VersionUtils.version); - conn.addRequestProperty(HttpHeaderConsts.USER_AGENT_HEADER, UtilsAndCommons.SERVER_VERSION); - conn.addRequestProperty(HttpHeaderConsts.REQUEST_SOURCE_HEADER, ApplicationUtils.getLocalAddress()); - } - - /** - * Encoding parameters. - * - * @param params parameters - * @param encoding charset - * @return parameters string - * @throws UnsupportedEncodingException unsupported encodin exception - */ - public static String encodingParams(Map params, String encoding) - throws UnsupportedEncodingException { - StringBuilder sb = new StringBuilder(); - if (null == params || params.isEmpty()) { - return null; - } - - params.put("encoding", encoding); - params.put("nofix", "1"); - - for (Map.Entry entry : params.entrySet()) { - if (StringUtils.isEmpty(entry.getValue())) { - continue; - } - - sb.append(entry.getKey()).append("="); - sb.append(URLEncoder.encode(entry.getValue(), encoding)); - sb.append("&"); - } - - return sb.toString(); - } - /** * Translate parameter map. * @@ -664,33 +365,4 @@ public class HttpClient { } return map; } - - public static class HttpResult { - - public final int code; - - public final String content; - - private final Map respHeaders; - - public HttpResult(int code, String content, Map respHeaders) { - this.code = code; - this.content = content; - this.respHeaders = respHeaders; - } - - public String getHeader(String name) { - return respHeaders.get(name); - } - } - - public static class HttpGetWithEntity extends HttpEntityEnclosingRequestBase { - - public static final String METHOD_NAME = "GET"; - - @Override - public String getMethod() { - return METHOD_NAME; - } - } } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/misc/HttpClientManager.java b/naming/src/main/java/com/alibaba/nacos/naming/misc/HttpClientManager.java new file mode 100644 index 000000000..bd1e3153c --- /dev/null +++ b/naming/src/main/java/com/alibaba/nacos/naming/misc/HttpClientManager.java @@ -0,0 +1,149 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.naming.misc; + +import com.alibaba.nacos.common.http.AbstractApacheHttpClientFactory; +import com.alibaba.nacos.common.http.AbstractHttpClientFactory; +import com.alibaba.nacos.common.http.HttpClientBeanHolder; +import com.alibaba.nacos.common.http.HttpClientConfig; +import com.alibaba.nacos.common.http.HttpClientFactory; +import com.alibaba.nacos.common.http.client.NacosAsyncRestTemplate; +import com.alibaba.nacos.common.http.client.NacosRestTemplate; +import com.alibaba.nacos.common.utils.ExceptionUtil; +import com.alibaba.nacos.common.utils.ThreadUtils; +import org.slf4j.Logger; + +import java.util.concurrent.TimeUnit; + +import static com.alibaba.nacos.naming.misc.Loggers.SRV_LOG; + +/** + * http Manager. + * + * @author mai.jh + */ +public class HttpClientManager { + + private static final int TIME_OUT_MILLIS = 10000; + + private static final int CON_TIME_OUT_MILLIS = 5000; + + private static final HttpClientFactory SYNC_HTTP_CLIENT_FACTORY = new SyncHttpClientFactory(); + + private static final HttpClientFactory ASYNC_HTTP_CLIENT_FACTORY = new AsyncHttpClientFactory(); + + private static final HttpClientFactory APACHE_SYNC_HTTP_CLIENT_FACTORY = new ApacheSyncHttpClientFactory(); + + private static final NacosRestTemplate NACOS_REST_TEMPLATE; + + private static final NacosRestTemplate APACHE_NACOS_REST_TEMPLATE; + + private static final NacosAsyncRestTemplate NACOS_ASYNC_REST_TEMPLATE; + + static { + // build nacos rest template + NACOS_REST_TEMPLATE = HttpClientBeanHolder.getNacosRestTemplate(SYNC_HTTP_CLIENT_FACTORY); + APACHE_NACOS_REST_TEMPLATE = HttpClientBeanHolder.getNacosRestTemplate(APACHE_SYNC_HTTP_CLIENT_FACTORY); + NACOS_ASYNC_REST_TEMPLATE = HttpClientBeanHolder.getNacosAsyncRestTemplate(ASYNC_HTTP_CLIENT_FACTORY); + + ThreadUtils.addShutdownHook(new Runnable() { + @Override + public void run() { + shutdown(); + } + }); + } + + public static NacosRestTemplate getNacosRestTemplate() { + return NACOS_REST_TEMPLATE; + } + + /** + * Use apache http client to achieve. + * @return NacosRestTemplate + */ + public static NacosRestTemplate getApacheRestTemplate() { + return APACHE_NACOS_REST_TEMPLATE; + } + + public static NacosAsyncRestTemplate getAsyncRestTemplate() { + return NACOS_ASYNC_REST_TEMPLATE; + } + + private static void shutdown() { + SRV_LOG.warn("[NamingServerHttpClientManager] Start destroying HTTP-Client"); + try { + HttpClientBeanHolder.shutdownNacostSyncRest(SYNC_HTTP_CLIENT_FACTORY.getClass().getName()); + HttpClientBeanHolder.shutdownNacostSyncRest(APACHE_SYNC_HTTP_CLIENT_FACTORY.getClass().getName()); + HttpClientBeanHolder.shutdownNacosAsyncRest(ASYNC_HTTP_CLIENT_FACTORY.getClass().getName()); + } catch (Exception ex) { + SRV_LOG.error("[NamingServerHttpClientManager] An exception occurred when the HTTP client was closed : {}", + ExceptionUtil.getStackTrace(ex)); + } + SRV_LOG.warn("[NamingServerHttpClientManager] Destruction of the end"); + } + + private static class AsyncHttpClientFactory extends AbstractHttpClientFactory { + + @Override + protected HttpClientConfig buildHttpClientConfig() { + return HttpClientConfig.builder().setConTimeOutMillis(CON_TIME_OUT_MILLIS) + .setReadTimeOutMillis(TIME_OUT_MILLIS) + .setUserAgent(UtilsAndCommons.SERVER_VERSION) + .setMaxConnTotal(-1) + .setMaxConnPerRoute(128) + .setMaxRedirects(0).build(); + } + + @Override + protected Logger assignLogger() { + return SRV_LOG; + } + } + + private static class SyncHttpClientFactory extends AbstractHttpClientFactory { + + @Override + protected HttpClientConfig buildHttpClientConfig() { + return HttpClientConfig.builder().setConTimeOutMillis(CON_TIME_OUT_MILLIS) + .setReadTimeOutMillis(TIME_OUT_MILLIS) + .setMaxRedirects(0).build(); + } + + @Override + protected Logger assignLogger() { + return SRV_LOG; + } + } + + private static class ApacheSyncHttpClientFactory extends AbstractApacheHttpClientFactory { + + @Override + protected HttpClientConfig buildHttpClientConfig() { + return HttpClientConfig.builder() + .setConnectionTimeToLive(500, TimeUnit.MILLISECONDS) + .setMaxConnTotal(Runtime.getRuntime().availableProcessors() * 2) + .setMaxConnPerRoute(Runtime.getRuntime().availableProcessors()) + .setMaxRedirects(0).build(); + } + + @Override + protected Logger assignLogger() { + return SRV_LOG; + } + } +} diff --git a/naming/src/main/java/com/alibaba/nacos/naming/misc/NamingProxy.java b/naming/src/main/java/com/alibaba/nacos/naming/misc/NamingProxy.java index 1225af7f1..bb08c2ac3 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/misc/NamingProxy.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/misc/NamingProxy.java @@ -17,11 +17,11 @@ package com.alibaba.nacos.naming.misc; import com.alibaba.nacos.common.constant.HttpHeaderConsts; +import com.alibaba.nacos.common.http.Callback; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.common.utils.VersionUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.ning.http.client.AsyncCompletionHandler; -import com.ning.http.client.Response; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.lang3.StringUtils; import java.io.IOException; @@ -69,28 +69,32 @@ public class NamingProxy { headers.put(HttpHeaderConsts.CLIENT_VERSION_HEADER, VersionUtils.version); headers.put(HttpHeaderConsts.USER_AGENT_HEADER, UtilsAndCommons.SERVER_VERSION); - headers.put("Connection", "Keep-Alive"); + headers.put(HttpHeaderConsts.CONNECTION, "Keep-Alive"); HttpClient.asyncHttpPutLarge( "http://" + server + ApplicationUtils.getContextPath() + UtilsAndCommons.NACOS_NAMING_CONTEXT + TIMESTAMP_SYNC_URL + "?source=" + NetUtils.localServer(), headers, checksums, - new AsyncCompletionHandler() { + new Callback() { @Override - public Object onCompleted(Response response) throws Exception { - if (HttpURLConnection.HTTP_OK != response.getStatusCode()) { + public void onReceive(RestResult result) { + if (!result.ok()) { Loggers.DISTRO.error("failed to req API: {}, code: {}, msg: {}", "http://" + server + ApplicationUtils.getContextPath() + UtilsAndCommons.NACOS_NAMING_CONTEXT + TIMESTAMP_SYNC_URL, - response.getStatusCode(), response.getResponseBody()); + result.getCode(), result.getMessage()); } - return null; } - + @Override - public void onThrowable(Throwable t) { + public void onError(Throwable throwable) { Loggers.DISTRO .error("failed to req API:" + "http://" + server + ApplicationUtils.getContextPath() - + UtilsAndCommons.NACOS_NAMING_CONTEXT + TIMESTAMP_SYNC_URL, t); + + UtilsAndCommons.NACOS_NAMING_CONTEXT + TIMESTAMP_SYNC_URL, throwable); + } + + @Override + public void onCancel() { + } }); } catch (Exception e) { @@ -110,17 +114,17 @@ public class NamingProxy { Map params = new HashMap<>(8); params.put("keys", StringUtils.join(keys, ",")); - HttpClient.HttpResult result = HttpClient.httpGetLarge( + RestResult result = HttpClient.httpGetLarge( "http://" + server + ApplicationUtils.getContextPath() + UtilsAndCommons.NACOS_NAMING_CONTEXT + DATA_GET_URL, new HashMap<>(8), JacksonUtils.toJson(params)); - if (HttpURLConnection.HTTP_OK == result.code) { - return result.content.getBytes(); + if (result.ok()) { + return result.getData().getBytes(); } throw new IOException("failed to req API: " + "http://" + server + ApplicationUtils.getContextPath() - + UtilsAndCommons.NACOS_NAMING_CONTEXT + DATA_GET_URL + ". code: " + result.code + " msg: " - + result.content); + + UtilsAndCommons.NACOS_NAMING_CONTEXT + DATA_GET_URL + ". code: " + result.getCode() + " msg: " + + result.getMessage()); } /** @@ -133,17 +137,17 @@ public class NamingProxy { public static byte[] getAllData(String server) throws Exception { Map params = new HashMap<>(8); - HttpClient.HttpResult result = HttpClient.httpGet( + RestResult result = HttpClient.httpGet( "http://" + server + ApplicationUtils.getContextPath() + UtilsAndCommons.NACOS_NAMING_CONTEXT + ALL_DATA_GET_URL, new ArrayList<>(), params); - if (HttpURLConnection.HTTP_OK == result.code) { - return result.content.getBytes(); + if (result.ok()) { + return result.getData().getBytes(); } throw new IOException("failed to req API: " + "http://" + server + ApplicationUtils.getContextPath() - + UtilsAndCommons.NACOS_NAMING_CONTEXT + DATA_GET_URL + ". code: " + result.code + " msg: " - + result.content); + + UtilsAndCommons.NACOS_NAMING_CONTEXT + ALL_DATA_GET_URL + ". code: " + result.getCode() + " msg: " + + result.getMessage()); } /** @@ -158,23 +162,23 @@ public class NamingProxy { headers.put(HttpHeaderConsts.CLIENT_VERSION_HEADER, VersionUtils.version); headers.put(HttpHeaderConsts.USER_AGENT_HEADER, UtilsAndCommons.SERVER_VERSION); - headers.put("Accept-Encoding", "gzip,deflate,sdch"); - headers.put("Connection", "Keep-Alive"); - headers.put("Content-Encoding", "gzip"); + headers.put(HttpHeaderConsts.ACCEPT_ENCODING, "gzip,deflate,sdch"); + headers.put(HttpHeaderConsts.CONNECTION, "Keep-Alive"); + headers.put(HttpHeaderConsts.CONTENT_ENCODING, "gzip"); try { - HttpClient.HttpResult result = HttpClient.httpPutLarge( + RestResult result = HttpClient.httpPutLarge( "http://" + curServer + ApplicationUtils.getContextPath() + UtilsAndCommons.NACOS_NAMING_CONTEXT + DATA_ON_SYNC_URL, headers, data); - if (HttpURLConnection.HTTP_OK == result.code) { + if (result.ok()) { return true; } - if (HttpURLConnection.HTTP_NOT_MODIFIED == result.code) { + if (HttpURLConnection.HTTP_NOT_MODIFIED == result.getCode()) { return true; } throw new IOException("failed to req API:" + "http://" + curServer + ApplicationUtils.getContextPath() - + UtilsAndCommons.NACOS_NAMING_CONTEXT + DATA_ON_SYNC_URL + ". code:" + result.code + " msg: " - + result.content); + + UtilsAndCommons.NACOS_NAMING_CONTEXT + DATA_ON_SYNC_URL + ". code:" + result.getCode() + " msg: " + + result.getData()); } catch (Exception e) { Loggers.SRV_LOG.warn("NamingProxy", e); } @@ -196,7 +200,7 @@ public class NamingProxy { HttpHeaderConsts.USER_AGENT_HEADER, UtilsAndCommons.SERVER_VERSION, "Accept-Encoding", "gzip,deflate,sdch", "Connection", "Keep-Alive", "Content-Encoding", "gzip"); - HttpClient.HttpResult result; + RestResult result; if (!curServer.contains(UtilsAndCommons.IP_PORT_SPLITER)) { curServer = curServer + UtilsAndCommons.IP_PORT_SPLITER + ApplicationUtils.getPort(); @@ -204,17 +208,17 @@ public class NamingProxy { result = HttpClient.httpGet("http://" + curServer + api, headers, params); - if (HttpURLConnection.HTTP_OK == result.code) { - return result.content; + if (result.ok()) { + return result.getData(); } - if (HttpURLConnection.HTTP_NOT_MODIFIED == result.code) { + if (HttpURLConnection.HTTP_NOT_MODIFIED == result.getCode()) { return StringUtils.EMPTY; } throw new IOException( - "failed to req API:" + "http://" + curServer + api + ". code:" + result.code + " msg: " - + result.content); + "failed to req API:" + "http://" + curServer + api + ". code:" + result.getCode() + " msg: " + + result.getMessage()); } catch (Exception e) { Loggers.SRV_LOG.warn("NamingProxy", e); } @@ -238,7 +242,7 @@ public class NamingProxy { HttpHeaderConsts.USER_AGENT_HEADER, UtilsAndCommons.SERVER_VERSION, "Accept-Encoding", "gzip,deflate,sdch", "Connection", "Keep-Alive", "Content-Encoding", "gzip"); - HttpClient.HttpResult result; + RestResult result; if (!curServer.contains(UtilsAndCommons.IP_PORT_SPLITER)) { curServer = curServer + UtilsAndCommons.IP_PORT_SPLITER + ApplicationUtils.getPort(); @@ -254,17 +258,17 @@ public class NamingProxy { + "/api/" + api, headers, params); } - if (HttpURLConnection.HTTP_OK == result.code) { - return result.content; + if (result.ok()) { + return result.getData(); } - if (HttpURLConnection.HTTP_NOT_MODIFIED == result.code) { + if (HttpURLConnection.HTTP_NOT_MODIFIED == result.getCode()) { return StringUtils.EMPTY; } throw new IOException("failed to req API:" + "http://" + curServer + ApplicationUtils.getContextPath() - + UtilsAndCommons.NACOS_NAMING_CONTEXT + "/api/" + api + ". code:" + result.code + " msg: " - + result.content); + + UtilsAndCommons.NACOS_NAMING_CONTEXT + "/api/" + api + ". code:" + result.getCode() + " msg: " + + result.getMessage()); } catch (Exception e) { Loggers.SRV_LOG.warn("NamingProxy", e); } @@ -288,7 +292,7 @@ public class NamingProxy { UtilsAndCommons.SERVER_VERSION, "Accept-Encoding", "gzip,deflate,sdch", "Connection", "Keep-Alive", "Content-Encoding", "gzip"); - HttpClient.HttpResult result; + RestResult result; if (!curServer.contains(UtilsAndCommons.IP_PORT_SPLITER)) { curServer = curServer + UtilsAndCommons.IP_PORT_SPLITER + ApplicationUtils.getPort(); @@ -304,17 +308,17 @@ public class NamingProxy { + path, headers, params); } - if (HttpURLConnection.HTTP_OK == result.code) { - return result.content; + if (result.ok()) { + return result.getData(); } - if (HttpURLConnection.HTTP_NOT_MODIFIED == result.code) { + if (HttpURLConnection.HTTP_NOT_MODIFIED == result.getCode()) { return StringUtils.EMPTY; } throw new IOException("failed to req API:" + "http://" + curServer + ApplicationUtils.getContextPath() - + UtilsAndCommons.NACOS_NAMING_CONTEXT + path + ". code:" + result.code + " msg: " - + result.content); + + UtilsAndCommons.NACOS_NAMING_CONTEXT + path + ". code:" + result.getCode() + " msg: " + + result.getMessage()); } catch (Exception e) { Loggers.SRV_LOG.warn("NamingProxy", e); } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/misc/NetUtils.java b/naming/src/main/java/com/alibaba/nacos/naming/misc/NetUtils.java index bcdf3f568..caaa48e1e 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/misc/NetUtils.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/misc/NetUtils.java @@ -16,8 +16,8 @@ package com.alibaba.nacos.naming.misc; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.InetUtils; /** * Net Utils. @@ -32,7 +32,7 @@ public class NetUtils { * @return local server address */ public static String localServer() { - return InetUtils.getSelfIp() + UtilsAndCommons.IP_PORT_SPLITER + ApplicationUtils.getPort(); + return InetUtils.getSelfIP() + UtilsAndCommons.IP_PORT_SPLITER + ApplicationUtils.getPort(); } /** diff --git a/naming/src/main/java/com/alibaba/nacos/naming/misc/ServerStatusSynchronizer.java b/naming/src/main/java/com/alibaba/nacos/naming/misc/ServerStatusSynchronizer.java index bb1243891..d16e7a2e3 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/misc/ServerStatusSynchronizer.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/misc/ServerStatusSynchronizer.java @@ -16,12 +16,11 @@ package com.alibaba.nacos.naming.misc; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.ning.http.client.AsyncCompletionHandler; -import com.ning.http.client.Response; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.common.http.Callback; +import com.alibaba.nacos.common.model.RestResult; import org.springframework.util.StringUtils; -import java.net.HttpURLConnection; import java.util.HashMap; import java.util.Map; @@ -52,15 +51,23 @@ public class ServerStatusSynchronizer implements Synchronizer { } try { - HttpClient.asyncHttpGet(url, null, params, new AsyncCompletionHandler() { + HttpClient.asyncHttpGet(url, null, params, new Callback() { @Override - public Integer onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { + public void onReceive(RestResult result) { + if (!result.ok()) { Loggers.SRV_LOG.warn("[STATUS-SYNCHRONIZE] failed to request serverStatus, remote server: {}", serverIP); - return 1; } - return 0; + } + + @Override + public void onError(Throwable throwable) { + Loggers.SRV_LOG.warn("[STATUS-SYNCHRONIZE] failed to request serverStatus, remote server: {}", serverIP, throwable); + } + + @Override + public void onCancel() { + } }); } catch (Exception e) { diff --git a/naming/src/main/java/com/alibaba/nacos/naming/misc/ServiceStatusSynchronizer.java b/naming/src/main/java/com/alibaba/nacos/naming/misc/ServiceStatusSynchronizer.java index c65fa93c2..2f03b14f9 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/misc/ServiceStatusSynchronizer.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/misc/ServiceStatusSynchronizer.java @@ -16,13 +16,12 @@ package com.alibaba.nacos.naming.misc; +import com.alibaba.nacos.common.http.Callback; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.utils.JacksonUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.ning.http.client.AsyncCompletionHandler; -import com.ning.http.client.Response; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.commons.lang3.StringUtils; -import java.net.HttpURLConnection; import java.util.HashMap; import java.util.Map; @@ -53,16 +52,24 @@ public class ServiceStatusSynchronizer implements Synchronizer { } try { - HttpClient.asyncHttpPostLarge(url, null, JacksonUtils.toJson(params), new AsyncCompletionHandler() { + HttpClient.asyncHttpPostLarge(url, null, JacksonUtils.toJson(params), new Callback() { @Override - public Integer onCompleted(Response response) throws Exception { - if (response.getStatusCode() != HttpURLConnection.HTTP_OK) { + public void onReceive(RestResult result) { + if (!result.ok()) { Loggers.SRV_LOG.warn("[STATUS-SYNCHRONIZE] failed to request serviceStatus, remote server: {}", serverIP); - - return 1; + } - return 0; + } + + @Override + public void onError(Throwable throwable) { + Loggers.SRV_LOG.warn("[STATUS-SYNCHRONIZE] failed to request serviceStatus, remote server: " + serverIP, throwable); + } + + @Override + public void onCancel() { + } }); } catch (Exception e) { diff --git a/naming/src/main/java/com/alibaba/nacos/naming/misc/UtilsAndCommons.java b/naming/src/main/java/com/alibaba/nacos/naming/misc/UtilsAndCommons.java index fb398bd83..05cdf0f57 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/misc/UtilsAndCommons.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/misc/UtilsAndCommons.java @@ -20,7 +20,7 @@ import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.api.selector.SelectorType; import com.alibaba.nacos.common.utils.JacksonUtils; import com.alibaba.nacos.common.utils.VersionUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.selector.LabelSelector; import com.alibaba.nacos.naming.selector.NoneSelector; import com.fasterxml.jackson.core.type.TypeReference; @@ -118,8 +118,6 @@ public class UtilsAndCommons { public static final String DATA_BASE_DIR = ApplicationUtils.getNacosHome() + File.separator + "data" + File.separator + "naming"; - public static final String RAFT_CACHE_FILE_SUFFIX = ".datum"; - public static final String RAFT_CACHE_FILE_PREFIX = "com.alibaba.nacos.naming"; public static final String NUMBER_PATTERN = "^\\d+$"; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/monitor/PerformanceLoggerThread.java b/naming/src/main/java/com/alibaba/nacos/naming/monitor/PerformanceLoggerThread.java index 98f7a20bc..369c12754 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/monitor/PerformanceLoggerThread.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/monitor/PerformanceLoggerThread.java @@ -16,6 +16,7 @@ package com.alibaba.nacos.naming.monitor; +import com.alibaba.nacos.naming.consistency.persistent.ClusterVersionJudgement; import com.alibaba.nacos.naming.consistency.persistent.raft.RaftCore; import com.alibaba.nacos.naming.consistency.persistent.raft.RaftPeer; import com.alibaba.nacos.naming.core.ServiceManager; @@ -48,6 +49,9 @@ public class PerformanceLoggerThread { @Autowired private RaftCore raftCore; + @Autowired + private ClusterVersionJudgement versionJudgement; + private static final long PERIOD = 5 * 60; @PostConstruct @@ -92,12 +96,22 @@ public class PerformanceLoggerThread { MetricsMonitor.getTotalPushMonitor().set(pushService.getTotalPush()); MetricsMonitor.getFailedPushMonitor().set(pushService.getFailedPushCount()); - if (raftCore.isLeader()) { - MetricsMonitor.getLeaderStatusMonitor().set(1); - } else if (raftCore.getPeerSet().local().state == RaftPeer.State.FOLLOWER) { - MetricsMonitor.getLeaderStatusMonitor().set(0); - } else { - MetricsMonitor.getLeaderStatusMonitor().set(2); + metricsRaftLeader(); + } + + /** + * Will deprecated after v1.4.x + */ + @Deprecated + private void metricsRaftLeader() { + if (!versionJudgement.allMemberIsNewVersion()) { + if (raftCore.isLeader()) { + MetricsMonitor.getLeaderStatusMonitor().set(1); + } else if (raftCore.getPeerSet().local().state == RaftPeer.State.FOLLOWER) { + MetricsMonitor.getLeaderStatusMonitor().set(0); + } else { + MetricsMonitor.getLeaderStatusMonitor().set(2); + } } } diff --git a/naming/src/main/java/com/alibaba/nacos/naming/selector/LabelSelector.java b/naming/src/main/java/com/alibaba/nacos/naming/selector/LabelSelector.java index f98615fba..dab5aa685 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/selector/LabelSelector.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/selector/LabelSelector.java @@ -22,7 +22,7 @@ import com.alibaba.nacos.api.selector.ExpressionSelector; import com.alibaba.nacos.api.selector.SelectorType; import com.alibaba.nacos.cmdb.service.CmdbReader; import com.alibaba.nacos.common.utils.JacksonUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.core.Instance; import com.fasterxml.jackson.annotation.JsonTypeInfo; import com.fasterxml.jackson.annotation.JsonTypeInfo.Id; diff --git a/naming/src/main/java/com/alibaba/nacos/naming/web/DistroFilter.java b/naming/src/main/java/com/alibaba/nacos/naming/web/DistroFilter.java index 579731e45..371182dd5 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/web/DistroFilter.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/web/DistroFilter.java @@ -19,6 +19,7 @@ package com.alibaba.nacos.naming.web; import com.alibaba.nacos.api.common.Constants; import com.alibaba.nacos.api.naming.CommonParams; import com.alibaba.nacos.common.constant.HttpHeaderConsts; +import com.alibaba.nacos.common.model.RestResult; import com.alibaba.nacos.common.utils.ExceptionUtil; import com.alibaba.nacos.common.utils.IoUtils; import com.alibaba.nacos.core.code.ControllerMethodsCache; @@ -139,11 +140,12 @@ public class DistroFilter implements Filter { final String body = IoUtils.toString(req.getInputStream(), Charsets.UTF_8.name()); final Map paramsValue = HttpClient.translateParameterMap(req.getParameterMap()); - HttpClient.HttpResult result = HttpClient + RestResult result = HttpClient .request("http://" + targetServer + req.getRequestURI(), headerList, paramsValue, body, PROXY_CONNECT_TIMEOUT, PROXY_READ_TIMEOUT, Charsets.UTF_8.name(), req.getMethod()); + String data = result.ok() ? result.getData() : result.getMessage(); try { - WebUtils.response(resp, result.content, result.code); + WebUtils.response(resp, data, result.getCode()); } catch (Exception ignore) { Loggers.SRV_LOG.warn("[DISTRO-FILTER] request failed: " + distroMapper.mapSrv(groupedServiceName) + urlString); diff --git a/naming/src/main/java/com/alibaba/nacos/naming/web/TrafficReviseFilter.java b/naming/src/main/java/com/alibaba/nacos/naming/web/TrafficReviseFilter.java index e0de5bc69..e738b7e74 100644 --- a/naming/src/main/java/com/alibaba/nacos/naming/web/TrafficReviseFilter.java +++ b/naming/src/main/java/com/alibaba/nacos/naming/web/TrafficReviseFilter.java @@ -17,7 +17,7 @@ package com.alibaba.nacos.naming.web; import com.alibaba.nacos.common.utils.HttpMethod; -import com.alibaba.nacos.core.utils.Constants; +import com.alibaba.nacos.sys.env.Constants; import com.alibaba.nacos.core.utils.WebUtils; import com.alibaba.nacos.naming.cluster.ServerStatus; import com.alibaba.nacos.naming.cluster.ServerStatusManager; diff --git a/naming/src/test/java/com/alibaba/nacos/naming/BaseTest.java b/naming/src/test/java/com/alibaba/nacos/naming/BaseTest.java index 9e36156e3..a81502b5b 100644 --- a/naming/src/test/java/com/alibaba/nacos/naming/BaseTest.java +++ b/naming/src/test/java/com/alibaba/nacos/naming/BaseTest.java @@ -16,7 +16,7 @@ package com.alibaba.nacos.naming; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.consistency.persistent.raft.RaftCore; import com.alibaba.nacos.naming.consistency.persistent.raft.RaftPeer; import com.alibaba.nacos.naming.consistency.persistent.raft.RaftPeerSet; diff --git a/naming/src/test/java/com/alibaba/nacos/naming/JraftTest.java b/naming/src/test/java/com/alibaba/nacos/naming/JraftTest.java new file mode 100644 index 000000000..b6f200422 --- /dev/null +++ b/naming/src/test/java/com/alibaba/nacos/naming/JraftTest.java @@ -0,0 +1,32 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.naming; + +import com.alibaba.nacos.consistency.serialize.HessianSerializer; +import com.alibaba.nacos.naming.core.Service; +import org.junit.Test; + +public class JraftTest { + + @Test + public void test() { + HessianSerializer serializer = new HessianSerializer(); + Service service = new Service(); + byte[] result = serializer.serialize(service); + Service service1 = serializer.deserialize(result, Service.class); + } +} diff --git a/naming/src/test/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroConsistencyServiceImplTest.java b/naming/src/test/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroConsistencyServiceImplTest.java index ffaa82a05..4a76dc1b4 100644 --- a/naming/src/test/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroConsistencyServiceImplTest.java +++ b/naming/src/test/java/com/alibaba/nacos/naming/consistency/ephemeral/distro/DistroConsistencyServiceImplTest.java @@ -24,9 +24,7 @@ import com.alibaba.nacos.naming.consistency.Datum; import com.alibaba.nacos.naming.consistency.KeyBuilder; import com.alibaba.nacos.naming.consistency.RecordListener; import com.alibaba.nacos.core.distributed.distro.DistroProtocol; -import com.alibaba.nacos.core.distributed.distro.component.DistroComponentHolder; import com.alibaba.nacos.core.distributed.distro.entity.DistroKey; -import com.alibaba.nacos.core.distributed.distro.task.DistroTaskEngineHolder; import com.alibaba.nacos.naming.core.Instances; import com.alibaba.nacos.naming.misc.GlobalConfig; import org.junit.After; @@ -41,7 +39,6 @@ import java.util.concurrent.ConcurrentLinkedQueue; import static org.junit.Assert.assertTrue; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; -import static org.mockito.Mockito.doReturn; import static org.mockito.Mockito.never; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; @@ -68,20 +65,12 @@ public class DistroConsistencyServiceImplTest extends BaseTest { @Mock private RecordListener recordListener; - @Mock - private DistroComponentHolder distroComponentHolder; - - @Mock - private DistroTaskEngineHolder distroTaskEngineHolder; - private Map> listeners; private Instances instances; @Before public void setUp() throws Exception { - doReturn(distroComponentHolder).when(context).getBean(DistroComponentHolder.class); - doReturn(distroTaskEngineHolder).when(context).getBean(DistroTaskEngineHolder.class); when(globalConfig.getTaskDispatchPeriod()).thenReturn(2000); distroConsistencyService = new DistroConsistencyServiceImpl(distroMapper, dataStore, serializer, switchDomain, globalConfig, distroProtocol); diff --git a/naming/src/test/java/com/alibaba/nacos/naming/consistency/persistent/ClusterVersionJudgementTest.java b/naming/src/test/java/com/alibaba/nacos/naming/consistency/persistent/ClusterVersionJudgementTest.java index 668ce22d9..e88d0245d 100644 --- a/naming/src/test/java/com/alibaba/nacos/naming/consistency/persistent/ClusterVersionJudgementTest.java +++ b/naming/src/test/java/com/alibaba/nacos/naming/consistency/persistent/ClusterVersionJudgementTest.java @@ -20,7 +20,7 @@ import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.MemberMetaDataConstants; import com.alibaba.nacos.core.cluster.NodeState; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils;; import org.junit.After; import org.junit.Assert; import org.junit.Before; diff --git a/naming/src/test/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingSnapshotOperationTest.java b/naming/src/test/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingSnapshotOperationTest.java index ccb73f6c8..47f3e900f 100644 --- a/naming/src/test/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingSnapshotOperationTest.java +++ b/naming/src/test/java/com/alibaba/nacos/naming/consistency/persistent/impl/NamingSnapshotOperationTest.java @@ -20,13 +20,14 @@ import com.alibaba.nacos.consistency.snapshot.Reader; import com.alibaba.nacos.consistency.snapshot.Writer; import com.alibaba.nacos.core.distributed.raft.RaftConfig; import com.alibaba.nacos.core.distributed.raft.utils.RaftExecutor; -import com.alibaba.nacos.core.storage.kv.RocksStorage; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.DiskUtils; +import com.alibaba.nacos.core.storage.kv.KvStorage; +import com.alibaba.nacos.sys.utils.ApplicationUtils;; +import com.alibaba.nacos.sys.utils.DiskUtils; import org.junit.After; import org.junit.Assert; import org.junit.Before; import org.junit.Test; +import org.mockito.Mock; import org.springframework.mock.env.MockEnvironment; import java.nio.file.Paths; @@ -42,7 +43,8 @@ public class NamingSnapshotOperationTest { ApplicationUtils.injectEnvironment(new MockEnvironment()); } - private RocksStorage storage; + @Mock + private KvStorage storage; private final String tmpDir = Paths.get(ApplicationUtils.getNacosTmpDir(), "rocks_test").toString(); @@ -53,7 +55,6 @@ public class NamingSnapshotOperationTest { @Before public void init() { DiskUtils.deleteQuietly(Paths.get(ApplicationUtils.getNacosTmpDir())); - storage = RocksStorage.createDefault("test", tmpDir); } @After @@ -81,4 +82,4 @@ public class NamingSnapshotOperationTest { Assert.assertTrue(res); } -} \ No newline at end of file +} diff --git a/naming/src/test/java/com/alibaba/nacos/naming/controllers/ClusterControllerTest.java b/naming/src/test/java/com/alibaba/nacos/naming/controllers/ClusterControllerTest.java index 3130f63e1..2e18a0a7f 100644 --- a/naming/src/test/java/com/alibaba/nacos/naming/controllers/ClusterControllerTest.java +++ b/naming/src/test/java/com/alibaba/nacos/naming/controllers/ClusterControllerTest.java @@ -52,6 +52,8 @@ public class ClusterControllerTest extends BaseTest { @Before public void before() { super.before(); + mockInjectSwitchDomain(); + mockInjectDistroMapper(); mockmvc = MockMvcBuilders.standaloneSetup(clusterController).build(); } @@ -87,17 +89,44 @@ public class ClusterControllerTest extends BaseTest { } @Test - public void testUpdateInvalidType() throws Exception { - expectedException.expectCause(isA(NacosException.class)); - expectedException.expectMessage("unknown health check type:{\"type\":\"123\"}"); + public void testUpdateHealthCheckerType() throws Exception { + Service service = new Service(TEST_SERVICE_NAME); service.setNamespaceId(Constants.DEFAULT_NAMESPACE_ID); when(serviceManager.getService(Constants.DEFAULT_NAMESPACE_ID, TEST_SERVICE_NAME)).thenReturn(service); + MockHttpServletRequestBuilder builder = MockMvcRequestBuilders .put(UtilsAndCommons.NACOS_NAMING_CONTEXT + "/cluster").param("clusterName", TEST_CLUSTER_NAME) .param("serviceName", TEST_SERVICE_NAME).param("healthChecker", "{\"type\":\"123\"}") .param("checkPort", "1").param("useInstancePort4Check", "true"); mockmvc.perform(builder); + + Assert.assertEquals("NONE", service.getClusterMap().get(TEST_CLUSTER_NAME).getHealthChecker().getType()); + + MockHttpServletRequestBuilder builder2 = MockMvcRequestBuilders + .put(UtilsAndCommons.NACOS_NAMING_CONTEXT + "/cluster").param("clusterName", TEST_CLUSTER_NAME) + .param("serviceName", TEST_SERVICE_NAME).param("healthChecker", "{\"type\":\"TCP\"}") + .param("checkPort", "1").param("useInstancePort4Check", "true"); + mockmvc.perform(builder2); + + Assert.assertEquals("TCP", service.getClusterMap().get(TEST_CLUSTER_NAME).getHealthChecker().getType()); + + MockHttpServletRequestBuilder builder3 = MockMvcRequestBuilders + .put(UtilsAndCommons.NACOS_NAMING_CONTEXT + "/cluster").param("clusterName", TEST_CLUSTER_NAME) + .param("serviceName", TEST_SERVICE_NAME).param("healthChecker", "{\"type\":\"HTTP\"}") + .param("checkPort", "1").param("useInstancePort4Check", "true"); + mockmvc.perform(builder3); + + Assert.assertEquals("HTTP", service.getClusterMap().get(TEST_CLUSTER_NAME).getHealthChecker().getType()); + + MockHttpServletRequestBuilder builder4 = MockMvcRequestBuilders + .put(UtilsAndCommons.NACOS_NAMING_CONTEXT + "/cluster").param("clusterName", TEST_CLUSTER_NAME) + .param("serviceName", TEST_SERVICE_NAME).param("healthChecker", "{\"type\":\"MYSQL\"}") + .param("checkPort", "1").param("useInstancePort4Check", "true"); + mockmvc.perform(builder4); + + Assert.assertEquals("MYSQL", service.getClusterMap().get(TEST_CLUSTER_NAME).getHealthChecker().getType()); + } } diff --git a/naming/src/test/java/com/alibaba/nacos/naming/core/ClusterTest.java b/naming/src/test/java/com/alibaba/nacos/naming/core/ClusterTest.java index 49a202b9d..f57c0474e 100644 --- a/naming/src/test/java/com/alibaba/nacos/naming/core/ClusterTest.java +++ b/naming/src/test/java/com/alibaba/nacos/naming/core/ClusterTest.java @@ -18,7 +18,7 @@ package com.alibaba.nacos.naming.core; import com.alibaba.nacos.api.naming.pojo.healthcheck.impl.Http; import com.alibaba.nacos.common.utils.JacksonUtils; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.naming.misc.SwitchDomain; import com.alibaba.nacos.naming.misc.SwitchDomain.TcpHealthParams; diff --git a/pom.xml b/pom.xml index e09e468c7..e365c3d21 100644 --- a/pom.xml +++ b/pom.xml @@ -126,7 +126,7 @@ 1.3.0 - 2.1.16.RELEASE + 2.1.17.RELEASE 3.0 2.6 3.4 @@ -579,6 +579,7 @@ istio consistency auth + sys @@ -686,6 +687,11 @@ nacos-auth ${project.version} + + ${project.groupId} + nacos-sys + ${project.version} + javax.servlet diff --git a/style/nacos-code-style-for-idea.xml b/style/nacos-code-style-for-idea.xml index 28846f8ae..6bd63a113 100644 --- a/style/nacos-code-style-for-idea.xml +++ b/style/nacos-code-style-for-idea.xml @@ -139,6 +139,7 @@

- * This method opens an InputStream for the file. - * When you have finished with the iterator you should close the stream - * to free internal resources. This can be done by calling the - * {@link org.apache.commons.io.LineIterator#close()} or - * {@link org.apache.commons.io.LineIterator#closeQuietly(org.apache.commons.io.LineIterator)} method. + * This method opens an InputStream for the file. When you have finished with the iterator you should + * close the stream to free internal resources. This can be done by calling the {@link + * org.apache.commons.io.LineIterator#close()} or {@link org.apache.commons.io.LineIterator#closeQuietly(org.apache.commons.io.LineIterator)} + * method. *

* The recommended usage pattern is: * 
@@ -487,12 +438,11 @@ public final class DiskUtils {
      * }
      *
*

- * If an exception occurs during the creation of the iterator, the - * underlying stream is closed. + * If an exception occurs during the creation of the iterator, the underlying stream is closed. *

* - * @param file the file to open for input, must not be null - * @param encoding the encoding to use, null means platform default + * @param file the file to open for input, must not be null + * @param encoding the encoding to use, null means platform default * @return an Iterator of the lines in the file, never null * @throws IOException in case of an I/O error (file closed) * @since 1.2 @@ -504,20 +454,20 @@ public final class DiskUtils { /** * Returns an Iterator for the lines in a File using the default encoding for the VM. * - * @param file the file to open for input, must not be null + * @param file the file to open for input, must not be null * @return an Iterator of the lines in the file, never null * @throws IOException in case of an I/O error (file closed) - * @since 1.3 * @see #lineIterator(File, String) + * @since 1.3 */ public static LineIterator lineIterator(File file) throws IOException { return new LineIterator(FileUtils.lineIterator(file, null)); } public static class LineIterator implements AutoCloseable { - + private final org.apache.commons.io.LineIterator target; - + /** * Constructs an iterator of the lines for a Reader. * @@ -526,7 +476,7 @@ public final class DiskUtils { LineIterator(org.apache.commons.io.LineIterator target) { this.target = target; } - + public boolean hasNext() { return target.hasNext(); } @@ -534,20 +484,20 @@ public final class DiskUtils { public String next() { return target.next(); } - + public String nextLine() { return target.nextLine(); } - + @Override public void close() { target.close(); } - + public void remove() { target.remove(); } - + public void forEachRemaining(Consumer action) { target.forEachRemaining(action); } diff --git a/core/src/main/java/com/alibaba/nacos/core/utils/InetUtils.java b/sys/src/main/java/com/alibaba/nacos/sys/utils/InetUtils.java similarity index 57% rename from core/src/main/java/com/alibaba/nacos/core/utils/InetUtils.java rename to sys/src/main/java/com/alibaba/nacos/sys/utils/InetUtils.java index e9241e825..65771d8cf 100644 --- a/core/src/main/java/com/alibaba/nacos/core/utils/InetUtils.java +++ b/sys/src/main/java/com/alibaba/nacos/sys/utils/InetUtils.java @@ -14,10 +14,11 @@ * limitations under the License. */ -package com.alibaba.nacos.core.utils; +package com.alibaba.nacos.sys.utils; import com.alibaba.nacos.common.notify.NotifyCenter; import com.alibaba.nacos.common.notify.SlowEvent; +import com.alibaba.nacos.sys.env.Constants; import org.apache.commons.lang3.StringUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -25,6 +26,7 @@ import org.slf4j.LoggerFactory; import java.io.IOException; import java.net.Inet4Address; import java.net.InetAddress; +import java.net.InetSocketAddress; import java.net.NetworkInterface; import java.net.UnknownHostException; import java.util.ArrayList; @@ -34,13 +36,11 @@ import java.util.Objects; import java.util.regex.Matcher; import java.util.regex.Pattern; -import static com.alibaba.nacos.core.utils.Constants.IGNORED_INTERFACES; -import static com.alibaba.nacos.core.utils.Constants.IP_ADDRESS; -import static com.alibaba.nacos.core.utils.Constants.NACOS_SERVER_IP; -import static com.alibaba.nacos.core.utils.Constants.PREFERRED_NETWORKS; -import static com.alibaba.nacos.core.utils.Constants.PREFER_HOSTNAME_OVER_IP; -import static com.alibaba.nacos.core.utils.Constants.SYSTEM_PREFER_HOSTNAME_OVER_IP; -import static com.alibaba.nacos.core.utils.Constants.USE_ONLY_SITE_INTERFACES; +import static com.alibaba.nacos.sys.env.Constants.IP_ADDRESS; +import static com.alibaba.nacos.sys.env.Constants.NACOS_SERVER_IP; +import static com.alibaba.nacos.sys.env.Constants.PREFER_HOSTNAME_OVER_IP; +import static com.alibaba.nacos.sys.env.Constants.SYSTEM_PREFER_HOSTNAME_OVER_IP; +import static com.alibaba.nacos.sys.env.Constants.USE_ONLY_SITE_INTERFACES; /** * Network card operation tool class. @@ -57,85 +57,82 @@ public class InetUtils { private static final Pattern IP_PATTERN = Pattern.compile(IP_REGEX); - private static String selfIp; + private static String selfIP; private static boolean useOnlySiteLocalInterface = false; - private static boolean preferHostnameOverIp = false; + private static boolean preferHostnameOverIP = false; - private static List preferredNetworks = new ArrayList(); + private static final List PREFERRED_NETWORKS = new ArrayList(); - private static List ignoredInterfaces = new ArrayList(); + private static final List IGNORED_INTERFACES = new ArrayList(); static { NotifyCenter.registerToSharePublisher(IPChangeEvent.class); - useOnlySiteLocalInterface = Boolean.parseBoolean(PropertyUtil.getProperty(USE_ONLY_SITE_INTERFACES)); + useOnlySiteLocalInterface = Boolean.parseBoolean(ApplicationUtils.getProperty(USE_ONLY_SITE_INTERFACES)); - List networks = PropertyUtil.getPropertyList(PREFERRED_NETWORKS); - preferredNetworks.addAll(networks); + List networks = ApplicationUtils.getPropertyList(Constants.PREFERRED_NETWORKS); + PREFERRED_NETWORKS.addAll(networks); - List interfaces = PropertyUtil.getPropertyList(IGNORED_INTERFACES); - ignoredInterfaces.addAll(interfaces); + List interfaces = ApplicationUtils.getPropertyList(Constants.IGNORED_INTERFACES); + IGNORED_INTERFACES.addAll(interfaces); final long delayMs = Long.getLong("nacos.core.inet.auto-refresh", 30_000L); Runnable ipAutoRefresh = new Runnable() { @Override public void run() { - try { - String nacosIp = System.getProperty(NACOS_SERVER_IP); - if (StringUtils.isBlank(nacosIp)) { - nacosIp = PropertyUtil.getProperty(IP_ADDRESS); - } - - if (!StringUtils.isBlank(nacosIp) && !isIP(nacosIp)) { - throw new RuntimeException("nacos address " + nacosIp + " is not ip"); - } - String tmpSelfIp = nacosIp; - if (StringUtils.isBlank(tmpSelfIp)) { - preferHostnameOverIp = Boolean.getBoolean(SYSTEM_PREFER_HOSTNAME_OVER_IP); - - if (!preferHostnameOverIp) { - preferHostnameOverIp = Boolean - .parseBoolean(PropertyUtil.getProperty(PREFER_HOSTNAME_OVER_IP)); - } - - if (preferHostnameOverIp) { - InetAddress inetAddress; - try { - inetAddress = InetAddress.getLocalHost(); - if (inetAddress.getHostName().equals(inetAddress.getCanonicalHostName())) { - tmpSelfIp = inetAddress.getHostName(); - } else { - tmpSelfIp = inetAddress.getCanonicalHostName(); - } - } catch (UnknownHostException ignore) { - LOG.warn("Unable to retrieve localhost"); - } - } else { - tmpSelfIp = Objects.requireNonNull(findFirstNonLoopbackAddress()).getHostAddress(); - } - } - - if (!Objects.equals(selfIp, tmpSelfIp) && Objects.nonNull(selfIp)) { - IPChangeEvent event = new IPChangeEvent(); - event.setOldIp(selfIp); - event.setNewIp(tmpSelfIp); - NotifyCenter.publishEvent(event); - } - selfIp = tmpSelfIp; - } finally { - GlobalExecutor.scheduleByCommon(this, delayMs); + String nacosIP = System.getProperty(NACOS_SERVER_IP); + if (StringUtils.isBlank(nacosIP)) { + nacosIP = ApplicationUtils.getProperty(IP_ADDRESS); } + + boolean illegalIP = !StringUtils.isBlank(nacosIP) && !(isIP(nacosIP) || isDomain(nacosIP)); + if (illegalIP) { + throw new RuntimeException("nacos address " + nacosIP + " is not ip"); + } + String tmpSelfIP = nacosIP; + if (StringUtils.isBlank(tmpSelfIP)) { + preferHostnameOverIP = Boolean.getBoolean(SYSTEM_PREFER_HOSTNAME_OVER_IP); + + if (!preferHostnameOverIP) { + preferHostnameOverIP = Boolean + .parseBoolean(ApplicationUtils.getProperty(PREFER_HOSTNAME_OVER_IP)); + } + + if (preferHostnameOverIP) { + InetAddress inetAddress; + try { + inetAddress = InetAddress.getLocalHost(); + if (inetAddress.getHostName().equals(inetAddress.getCanonicalHostName())) { + tmpSelfIP = inetAddress.getHostName(); + } else { + tmpSelfIP = inetAddress.getCanonicalHostName(); + } + } catch (UnknownHostException ignore) { + LOG.warn("Unable to retrieve localhost"); + } + } else { + tmpSelfIP = Objects.requireNonNull(findFirstNonLoopbackAddress()).getHostAddress(); + } + } + + if (!Objects.equals(selfIP, tmpSelfIP) && Objects.nonNull(selfIP)) { + IPChangeEvent event = new IPChangeEvent(); + event.setOldIP(selfIP); + event.setNewIP(tmpSelfIP); + NotifyCenter.publishEvent(event); + } + selfIP = tmpSelfIP; } }; - GlobalExecutor.runWithoutThread(ipAutoRefresh); + ipAutoRefresh.run(); } - public static String getSelfIp() { - return selfIp; + public static String getSelfIP() { + return selfIP; } /** @@ -196,10 +193,10 @@ public class InetUtils { } return siteLocalAddress; } - if (preferredNetworks.isEmpty()) { + if (PREFERRED_NETWORKS.isEmpty()) { return true; } - for (String regex : preferredNetworks) { + for (String regex : PREFERRED_NETWORKS) { final String hostAddress = address.getHostAddress(); if (hostAddress.matches(regex) || hostAddress.startsWith(regex)) { return true; @@ -210,7 +207,7 @@ public class InetUtils { } private static boolean ignoreInterface(String interfaceName) { - for (String regex : ignoredInterfaces) { + for (String regex : IGNORED_INTERFACES) { if (interfaceName.matches(regex)) { LOG.debug("Ignoring interface: " + interfaceName); return true; @@ -224,35 +221,50 @@ public class InetUtils { return matcher.matches(); } + /** + * juege str is right domain. + * + * @param str nacosIP + * @return nacosIP is domain + */ + public static boolean isDomain(String str) { + InetSocketAddress address = new InetSocketAddress(str, 0); + boolean unResolved = address.isUnresolved(); + if (unResolved) { + LOG.warn("the domain: '" + str + "' can not be resolved"); + } + return !unResolved; + } + /** * {@link com.alibaba.nacos.core.cluster.ServerMemberManager} is listener. */ @SuppressWarnings({"PMD.ClassNamingShouldBeCamelRule", "checkstyle:AbbreviationAsWordInName"}) public static class IPChangeEvent extends SlowEvent { - private String oldIp; + private String oldIP; - private String newIp; + private String newIP; - public String getOldIp() { - return oldIp; + public String getOldIP() { + return oldIP; } - public void setOldIp(String oldIp) { - this.oldIp = oldIp; + public void setOldIP(String oldIP) { + this.oldIP = oldIP; } - public String getNewIp() { - return newIp; + public String getNewIP() { + return newIP; } - public void setNewIp(String newIp) { - this.newIp = newIp; + public void setNewIP(String newIP) { + this.newIP = newIP; } @Override public String toString() { - return "IPChangeEvent{" + "oldIp='" + oldIp + '\'' + ", newIp='" + newIp + '\'' + '}'; + return "IPChangeEvent{" + "oldIP='" + oldIP + '\'' + ", newIP='" + newIP + '\'' + '}'; } } diff --git a/core/src/main/resources/META-INF/nacos-default.properties b/sys/src/main/resources/META-INF/nacos-default.properties similarity index 85% rename from core/src/main/resources/META-INF/nacos-default.properties rename to sys/src/main/resources/META-INF/nacos-default.properties index 0eda3de14..6e081a98e 100644 --- a/core/src/main/resources/META-INF/nacos-default.properties +++ b/sys/src/main/resources/META-INF/nacos-default.properties @@ -32,5 +32,5 @@ spring.messages.encoding=UTF-8 ## Exclude Spring Boot Auto-Configuration class(es) spring.autoconfigure.exclude=\ -org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration,\ -org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration + org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration,\ + org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration \ No newline at end of file diff --git a/sys/src/main/resources/META-INF/spring.factories b/sys/src/main/resources/META-INF/spring.factories new file mode 100644 index 000000000..71a4ccbfc --- /dev/null +++ b/sys/src/main/resources/META-INF/spring.factories @@ -0,0 +1,9 @@ +# PropertySource Loaders +org.springframework.boot.env.PropertySourceLoader=\ + com.alibaba.nacos.sys.env.NacosAutoRefreshPropertySourceLoader +# EnvironmentPostProcessor +org.springframework.boot.env.EnvironmentPostProcessor=\ +com.alibaba.nacos.sys.env.NacosDefaultPropertySourceEnvironmentPostProcessor +# ApplicationContextInitializer +org.springframework.context.ApplicationContextInitializer=\ + com.alibaba.nacos.sys.utils.ApplicationUtils \ No newline at end of file diff --git a/sys/src/test/java/com/alibaba/nacos/sys/env/NacosAutoRefreshPropertySourceLoaderTest.java b/sys/src/test/java/com/alibaba/nacos/sys/env/NacosAutoRefreshPropertySourceLoaderTest.java new file mode 100644 index 000000000..8d3720f9b --- /dev/null +++ b/sys/src/test/java/com/alibaba/nacos/sys/env/NacosAutoRefreshPropertySourceLoaderTest.java @@ -0,0 +1,92 @@ +/* + * Copyright 1999-2018 Alibaba Group Holding Ltd. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.alibaba.nacos.sys.env; + +import com.alibaba.nacos.api.exception.NacosException; +import com.alibaba.nacos.common.utils.ByteUtils; +import com.alibaba.nacos.common.utils.ThreadUtils; +import com.alibaba.nacos.sys.file.FileChangeEvent; +import com.alibaba.nacos.sys.file.FileWatcher; +import com.alibaba.nacos.sys.file.WatchFileCenter; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; +import org.apache.commons.lang3.StringUtils; +import org.junit.AfterClass; +import org.junit.Assert; +import org.junit.BeforeClass; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.core.env.ConfigurableEnvironment; +import org.springframework.test.context.junit4.SpringRunner; + +import java.io.File; +import java.io.IOException; +import java.net.URISyntaxException; +import java.net.URL; +import java.util.concurrent.CountDownLatch; + +@RunWith(SpringRunner.class) +@SpringBootTest(classes = NacosAutoRefreshPropertySourceLoaderTest.class, webEnvironment = SpringBootTest.WebEnvironment.NONE) +public class NacosAutoRefreshPropertySourceLoaderTest { + + @Autowired + private ConfigurableEnvironment environment; + + private static String oldConfPath = ""; + + @BeforeClass + public static void before() throws URISyntaxException { + oldConfPath = ApplicationUtils.getConfFilePath(); + ApplicationUtils.setConfFilePath(new File(ClassLoader.getSystemResource("application.properties").toURI()).getParent()); + } + + @AfterClass + public static void after() { + ApplicationUtils.setConfFilePath(oldConfPath); + } + + @Test + public void testConfigFileAutoRefresh() throws URISyntaxException, InterruptedException, NacosException, IOException { + final URL url = ClassLoader.getSystemResource("application.properties"); + ApplicationUtils.setContextPath(url.getPath()); + final String val1 = environment.getProperty("name"); + Assert.assertEquals("test-1", val1); + final File file = new File(url.toURI()); + final String newKey = "nacos.config.refresh-" + System.currentTimeMillis(); + final String newVal = System.currentTimeMillis() + "-lessspring"; + DiskUtils.writeFile(file, ByteUtils.toBytes("\n" + newKey + "=" + newVal), true); + CountDownLatch latch = new CountDownLatch(1); + WatchFileCenter.registerWatcher(ApplicationUtils.getConfFilePath(), new FileWatcher() { + @Override + public void onChange(FileChangeEvent event) { + latch.countDown(); + } + + @Override + public boolean interest(String context) { + return StringUtils.contains(context, "application.properties"); + } + }); + latch.await(); + ThreadUtils.sleep(10_000); + final String val2 = environment.getProperty(newKey); + Assert.assertEquals(newVal, val2); + } + +} \ No newline at end of file diff --git a/core/src/test/java/com/alibaba/nacos/core/env/NacosDefaultPropertySourceEnvironmentPostProcessorTest.java b/sys/src/test/java/com/alibaba/nacos/sys/env/NacosDefaultPropertySourceEnvironmentPostProcessorTest.java similarity index 93% rename from core/src/test/java/com/alibaba/nacos/core/env/NacosDefaultPropertySourceEnvironmentPostProcessorTest.java rename to sys/src/test/java/com/alibaba/nacos/sys/env/NacosDefaultPropertySourceEnvironmentPostProcessorTest.java index ef9846240..2223cf864 100644 --- a/core/src/test/java/com/alibaba/nacos/core/env/NacosDefaultPropertySourceEnvironmentPostProcessorTest.java +++ b/sys/src/test/java/com/alibaba/nacos/sys/env/NacosDefaultPropertySourceEnvironmentPostProcessorTest.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.alibaba.nacos.core.env; +package com.alibaba.nacos.sys.env; import org.junit.Assert; import org.junit.Test; @@ -29,8 +29,8 @@ import org.springframework.test.context.junit4.SpringRunner; import java.util.HashSet; -import static com.alibaba.nacos.core.env.NacosDefaultPropertySourceEnvironmentPostProcessor.PROPERTY_SOURCE_NAME; -import static com.alibaba.nacos.core.env.NacosDefaultPropertySourceEnvironmentPostProcessor.RESOURCE_LOCATION_PATTERN; +import static com.alibaba.nacos.sys.env.NacosDefaultPropertySourceEnvironmentPostProcessor.PROPERTY_SOURCE_NAME; +import static com.alibaba.nacos.sys.env.NacosDefaultPropertySourceEnvironmentPostProcessor.RESOURCE_LOCATION_PATTERN; import static java.util.Arrays.asList; /** diff --git a/sys/src/test/resources/application.properties b/sys/src/test/resources/application.properties new file mode 100644 index 000000000..aaba4fabc --- /dev/null +++ b/sys/src/test/resources/application.properties @@ -0,0 +1,17 @@ +# +# Copyright 1999-2018 Alibaba Group Holding Ltd. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +name=test-1 \ No newline at end of file diff --git a/test/src/test/java/com/alibaba/nacos/test/common/WatchFileCenter_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/common/WatchFileCenter_ITCase.java index 044e0e8ee..909fbe64e 100644 --- a/test/src/test/java/com/alibaba/nacos/test/common/WatchFileCenter_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/common/WatchFileCenter_ITCase.java @@ -16,12 +16,12 @@ package com.alibaba.nacos.test.common; -import com.alibaba.nacos.core.file.FileChangeEvent; -import com.alibaba.nacos.core.file.FileWatcher; -import com.alibaba.nacos.core.file.WatchFileCenter; +import com.alibaba.nacos.sys.file.FileChangeEvent; +import com.alibaba.nacos.sys.file.FileWatcher; +import com.alibaba.nacos.sys.file.WatchFileCenter; import com.alibaba.nacos.common.utils.ByteUtils; import com.alibaba.nacos.common.utils.ConcurrentHashSet; -import com.alibaba.nacos.core.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; import com.alibaba.nacos.common.utils.ThreadUtils; import org.apache.commons.lang3.StringUtils; import org.junit.AfterClass; @@ -30,7 +30,6 @@ import org.junit.BeforeClass; import org.junit.Test; import java.io.File; -import java.io.IOException; import java.nio.charset.StandardCharsets; import java.nio.file.Paths; import java.util.Set; @@ -46,143 +45,138 @@ import java.util.function.Consumer; * @author liaochuntao */ public class WatchFileCenter_ITCase { - - static final String path = Paths - .get(System.getProperty("user.home"), "/watch_file_change_test").toString(); - - final Object monitor = new Object(); - - static final Executor executor = Executors.newFixedThreadPool(32); - - @BeforeClass - public static void beforeCls() throws Exception { - DiskUtils.deleteDirThenMkdir(path); - } - - @AfterClass - public static void afterCls() throws Exception { - DiskUtils.deleteDirectory(path); - } - - // The last file change must be notified - - @Test - public void test_high_concurrency_modify() throws Exception { - AtomicInteger count = new AtomicInteger(0); - Set set = new ConcurrentHashSet<>(); - - final String fileName = "test2_file_change"; - final File file = Paths.get(path, fileName).toFile(); - - func(fileName, file, content -> { - set.add(content); - count.incrementAndGet(); - }); - - ThreadUtils.sleep(5_000L); - } - - @Test - public void test_modify_file_much() throws Exception { - final String fileName = "modify_file_much"; - final File file = Paths.get(path, fileName).toFile(); - - CountDownLatch latch = new CountDownLatch(3); - AtomicInteger count = new AtomicInteger(0); - - WatchFileCenter.registerWatcher(path, new FileWatcher() { - @Override - public void onChange(FileChangeEvent event) { - try { - System.out.println(event); - System.out.println(DiskUtils.readFile(file)); - count.incrementAndGet(); - } finally { - latch.countDown(); - } - } - - @Override - public boolean interest(String context) { - return StringUtils.contains(context, fileName); - } - }); - - for (int i = 0; i < 3; i ++) { - DiskUtils.writeFile(file, ByteUtils.toBytes(("test_modify_file_" + i)), false); - ThreadUtils.sleep(10_000L); - } - - latch.await(10_000L, TimeUnit.MILLISECONDS); - - Assert.assertEquals(3, count.get()); - } - - @Test - public void test_multi_file_modify() throws Exception { - CountDownLatch latch = new CountDownLatch(10); - for (int i = 0; i < 10; i++) { - AtomicInteger count = new AtomicInteger(0); - Set set = new ConcurrentHashSet<>(); - - final String fileName = "test2_file_change_" + i; - final File file = Paths.get(path, fileName).toFile(); - - executor.execute(() -> { - try { - func(fileName, file, content -> { - set.add(content); - count.incrementAndGet(); - }); - } catch (Throwable ex) { - ex.printStackTrace(); - } finally { - latch.countDown(); - } - }); - } - latch.await(10_000L, TimeUnit.MILLISECONDS); - - ThreadUtils.sleep(5_000L); - } - - private void func(final String fileName, final File file, - final Consumer consumer) throws Exception { - CountDownLatch latch = new CountDownLatch(100); - DiskUtils.touch(file); - WatchFileCenter.registerWatcher(path, new FileWatcher() { - @Override - public void onChange(FileChangeEvent event) { - final File file = Paths.get(path, fileName).toFile(); - final String content = DiskUtils.readFile(file); - consumer.accept(content); - } - - @Override - public boolean interest(String context) { - return StringUtils.contains(context, fileName); - } - }); - - final AtomicInteger id = new AtomicInteger(0); - final AtomicReference finalContent = new AtomicReference<>(null); - for (int i = 0; i < 100; i++) { - executor.execute(() -> { - final String j = fileName + "_" + id.incrementAndGet(); - try { - final File file1 = Paths.get(path, fileName).toFile(); - synchronized (monitor) { - finalContent.set(j); - DiskUtils.writeFile(file1, j.getBytes(StandardCharsets.UTF_8), - false); - } - } catch (IOException e) { - Assert.fail(e.getMessage()); - } finally { - latch.countDown(); - } - }); - } - } - + + static final String path = Paths.get(System.getProperty("user.home"), "/watch_file_change_test").toString(); + + final Object monitor = new Object(); + + static final Executor executor = Executors.newFixedThreadPool(32); + + @BeforeClass + public static void beforeCls() throws Exception { + DiskUtils.deleteDirThenMkdir(path); + } + + @AfterClass + public static void afterCls() throws Exception { + DiskUtils.deleteDirectory(path); + } + + // The last file change must be notified + + @Test + public void test_high_concurrency_modify() throws Exception { + AtomicInteger count = new AtomicInteger(0); + Set set = new ConcurrentHashSet<>(); + + final String fileName = "test2_file_change"; + final File file = Paths.get(path, fileName).toFile(); + + func(fileName, file, content -> { + set.add(content); + count.incrementAndGet(); + }); + + ThreadUtils.sleep(5_000L); + } + + @Test + public void test_modify_file_much() throws Exception { + final String fileName = "modify_file_much"; + final File file = Paths.get(path, fileName).toFile(); + + CountDownLatch latch = new CountDownLatch(3); + AtomicInteger count = new AtomicInteger(0); + + WatchFileCenter.registerWatcher(path, new FileWatcher() { + @Override + public void onChange(FileChangeEvent event) { + try { + System.out.println(event); + System.out.println(DiskUtils.readFile(file)); + count.incrementAndGet(); + } finally { + latch.countDown(); + } + } + + @Override + public boolean interest(String context) { + return StringUtils.contains(context, fileName); + } + }); + + for (int i = 0; i < 3; i++) { + DiskUtils.writeFile(file, ByteUtils.toBytes(("test_modify_file_" + i)), false); + ThreadUtils.sleep(10_000L); + } + + latch.await(10_000L, TimeUnit.MILLISECONDS); + + Assert.assertEquals(3, count.get()); + } + + @Test + public void test_multi_file_modify() throws Exception { + CountDownLatch latch = new CountDownLatch(10); + for (int i = 0; i < 10; i++) { + AtomicInteger count = new AtomicInteger(0); + Set set = new ConcurrentHashSet<>(); + + final String fileName = "test2_file_change_" + i; + final File file = Paths.get(path, fileName).toFile(); + + executor.execute(() -> { + try { + func(fileName, file, content -> { + set.add(content); + count.incrementAndGet(); + }); + } catch (Throwable ex) { + ex.printStackTrace(); + } finally { + latch.countDown(); + } + }); + } + latch.await(10_000L, TimeUnit.MILLISECONDS); + + ThreadUtils.sleep(5_000L); + } + + private void func(final String fileName, final File file, final Consumer consumer) throws Exception { + CountDownLatch latch = new CountDownLatch(100); + DiskUtils.touch(file); + WatchFileCenter.registerWatcher(path, new FileWatcher() { + @Override + public void onChange(FileChangeEvent event) { + final File file = Paths.get(path, fileName).toFile(); + final String content = DiskUtils.readFile(file); + consumer.accept(content); + } + + @Override + public boolean interest(String context) { + return StringUtils.contains(context, fileName); + } + }); + + final AtomicInteger id = new AtomicInteger(0); + final AtomicReference finalContent = new AtomicReference<>(null); + for (int i = 0; i < 100; i++) { + executor.execute(() -> { + final String j = fileName + "_" + id.incrementAndGet(); + try { + final File file1 = Paths.get(path, fileName).toFile(); + synchronized (monitor) { + finalContent.set(j); + DiskUtils.writeFile(file1, j.getBytes(StandardCharsets.UTF_8), false); + } + } finally { + latch.countDown(); + } + }); + } + } + } diff --git a/test/src/test/java/com/alibaba/nacos/test/config/ConfigCleanUtils.java b/test/src/test/java/com/alibaba/nacos/test/config/ConfigCleanUtils.java index ffc652a4e..5bb1081a2 100644 --- a/test/src/test/java/com/alibaba/nacos/test/config/ConfigCleanUtils.java +++ b/test/src/test/java/com/alibaba/nacos/test/config/ConfigCleanUtils.java @@ -17,7 +17,7 @@ package com.alibaba.nacos.test.config; import com.alibaba.nacos.client.config.impl.LocalConfigInfoProcessor; -import com.alibaba.nacos.core.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; import java.io.IOException; diff --git a/test/src/test/java/com/alibaba/nacos/test/config/ConfigDerbyImport_CITCase.java b/test/src/test/java/com/alibaba/nacos/test/config/ConfigDerbyImport_CITCase.java index 0f14a13dc..b3c3a98c1 100644 --- a/test/src/test/java/com/alibaba/nacos/test/config/ConfigDerbyImport_CITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/config/ConfigDerbyImport_CITCase.java @@ -23,11 +23,8 @@ import com.alibaba.nacos.common.utils.ByteUtils; import com.alibaba.nacos.config.server.model.ConfigInfo; import com.alibaba.nacos.config.server.service.repository.PersistService; import com.alibaba.nacos.config.server.service.repository.embedded.DatabaseOperate; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.DiskUtils; -import org.junit.AfterClass; +import com.alibaba.nacos.sys.utils.DiskUtils; import org.junit.Assert; -import org.junit.BeforeClass; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; @@ -36,7 +33,6 @@ import org.springframework.context.ApplicationContext; import org.springframework.test.context.junit4.SpringRunner; import java.io.File; -import java.nio.file.Paths; import java.util.concurrent.CompletableFuture; /** diff --git a/test/src/test/java/com/alibaba/nacos/test/config/ConfigDerbyRaft_DITCase.java b/test/src/test/java/com/alibaba/nacos/test/config/ConfigDerbyRaft_DITCase.java index 322494a4a..95e41ce61 100644 --- a/test/src/test/java/com/alibaba/nacos/test/config/ConfigDerbyRaft_DITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/config/ConfigDerbyRaft_DITCase.java @@ -32,7 +32,7 @@ import com.alibaba.nacos.consistency.cp.CPProtocol; import com.alibaba.nacos.core.distributed.id.IdGeneratorManager; import com.alibaba.nacos.core.distributed.raft.utils.JRaftConstants; import com.alibaba.nacos.core.utils.GenericType; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import com.alibaba.nacos.test.core.BaseClusterTest; import org.junit.Assert; @@ -357,7 +357,7 @@ public class ConfigDerbyRaft_DITCase extends BaseClusterTest { // transfer leader to ip:8807 Map transfer = new HashMap<>(); - transfer.put(JRaftConstants.TRANSFER_LEADER, InetUtils.getSelfIp() + ":9847"); + transfer.put(JRaftConstants.TRANSFER_LEADER, InetUtils.getSelfIP() + ":9847"); RestResult result = protocol7.execute(transfer); System.out.println(result); Assert.assertTrue(result.ok()); @@ -372,7 +372,7 @@ public class ConfigDerbyRaft_DITCase extends BaseClusterTest { // transfer leader to ip:8808 transfer = new HashMap<>(); - transfer.put(JRaftConstants.TRANSFER_LEADER, InetUtils.getSelfIp() + ":9848"); + transfer.put(JRaftConstants.TRANSFER_LEADER, InetUtils.getSelfIP() + ":9848"); result = protocol8.execute(transfer); System.out.println(result); Assert.assertTrue(result.ok()); @@ -387,7 +387,7 @@ public class ConfigDerbyRaft_DITCase extends BaseClusterTest { // transfer leader to ip:8809 transfer = new HashMap<>(); - transfer.put(JRaftConstants.TRANSFER_LEADER, InetUtils.getSelfIp() + ":9849"); + transfer.put(JRaftConstants.TRANSFER_LEADER, InetUtils.getSelfIP() + ":9849"); result = protocol9.execute(transfer); System.out.println(result); Assert.assertTrue(result.ok()); diff --git a/test/src/test/java/com/alibaba/nacos/test/core/BaseClusterTest.java b/test/src/test/java/com/alibaba/nacos/test/core/BaseClusterTest.java index c1f79663c..def842773 100644 --- a/test/src/test/java/com/alibaba/nacos/test/core/BaseClusterTest.java +++ b/test/src/test/java/com/alibaba/nacos/test/core/BaseClusterTest.java @@ -21,18 +21,17 @@ import com.alibaba.nacos.api.PropertyKeyConst; import com.alibaba.nacos.api.config.ConfigService; import com.alibaba.nacos.api.naming.NamingService; import com.alibaba.nacos.common.http.HttpClientBeanHolder; -import com.alibaba.nacos.common.http.NSyncHttpClient; import com.alibaba.nacos.common.http.client.NacosRestTemplate; import com.alibaba.nacos.common.notify.Event; import com.alibaba.nacos.common.notify.NotifyCenter; import com.alibaba.nacos.common.notify.listener.Subscriber; -import com.alibaba.nacos.core.utils.DiskUtils; import com.alibaba.nacos.config.server.model.event.RaftDbErrorEvent; import com.alibaba.nacos.config.server.service.repository.embedded.DistributedDatabaseOperateImpl; import com.alibaba.nacos.consistency.cp.CPProtocol; import com.alibaba.nacos.consistency.cp.MetadataKey; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import com.alibaba.nacos.test.base.HttpClient4Test; import org.junit.AfterClass; @@ -91,7 +90,7 @@ public class BaseClusterTest extends HttpClient4Test { static { System.getProperties().setProperty("nacos.core.auth.enabled", "false"); System.getProperties().setProperty("embeddedStorage", "true"); - String ip = InetUtils.getSelfIp(); + String ip = InetUtils.getSelfIP(); clusterInfo = "nacos.member.list=" + ip + ":8847," + ip + ":8848," + ip + ":8849"; NotifyCenter.registerSubscriber(new Subscriber() { diff --git a/test/src/test/java/com/alibaba/nacos/test/core/InetUtils_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/core/InetUtils_ITCase.java index 1b219072b..f1c520977 100644 --- a/test/src/test/java/com/alibaba/nacos/test/core/InetUtils_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/core/InetUtils_ITCase.java @@ -19,7 +19,7 @@ package com.alibaba.nacos.test.core; import com.alibaba.nacos.common.notify.Event; import com.alibaba.nacos.common.notify.NotifyCenter; import com.alibaba.nacos.common.notify.listener.Subscriber; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import org.junit.Assert; import org.junit.Test; @@ -28,7 +28,8 @@ import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicReference; -import static com.alibaba.nacos.core.utils.Constants.NACOS_SERVER_IP; +import static com.alibaba.nacos.sys.env.Constants.NACOS_SERVER_IP; + /** * @author liaochuntao @@ -38,7 +39,7 @@ public class InetUtils_ITCase { static { System.setProperty("nacos.core.inet.auto-refresh", "3"); // For load InetUtils.class - InetUtils.getSelfIp(); + InetUtils.getSelfIP(); } @Test @@ -52,10 +53,10 @@ public class InetUtils_ITCase { Subscriber subscribe = new Subscriber() { @Override public void onEvent(InetUtils.IPChangeEvent event) { - if (Objects.nonNull(event.getOldIp())) { + if (Objects.nonNull(event.getOldIP())) { try { System.out.println(event); - reference.set(event.getNewIp()); + reference.set(event.getNewIP()); } finally { latch.countDown(); @@ -73,7 +74,7 @@ public class InetUtils_ITCase { latch.await(10_000L, TimeUnit.MILLISECONDS); Assert.assertEquals(testIp, reference.get()); - Assert.assertEquals(testIp, InetUtils.getSelfIp()); + Assert.assertEquals(testIp, InetUtils.getSelfIP()); } } diff --git a/test/src/test/java/com/alibaba/nacos/test/core/SnowFlowerIdGenerator_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/core/SnowFlowerIdGenerator_ITCase.java index 002695ecd..573b7c61c 100644 --- a/test/src/test/java/com/alibaba/nacos/test/core/SnowFlowerIdGenerator_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/core/SnowFlowerIdGenerator_ITCase.java @@ -17,7 +17,7 @@ package com.alibaba.nacos.test.core; import com.alibaba.nacos.core.distributed.id.SnowFlowerIdGenerator; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Test; import org.springframework.core.env.StandardEnvironment; diff --git a/test/src/test/java/com/alibaba/nacos/test/core/auth/NamingAuth_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/core/auth/NamingAuth_ITCase.java index 8c959fbfa..707feea32 100644 --- a/test/src/test/java/com/alibaba/nacos/test/core/auth/NamingAuth_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/core/auth/NamingAuth_ITCase.java @@ -21,7 +21,7 @@ import com.alibaba.nacos.api.PropertyKeyConst; import com.alibaba.nacos.api.exception.NacosException; import com.alibaba.nacos.api.naming.NamingService; import com.alibaba.nacos.api.naming.pojo.Instance; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.apache.http.HttpStatus; import org.junit.After; import org.junit.Assert; diff --git a/test/src/test/java/com/alibaba/nacos/test/core/cluster/MemberLookup_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/core/cluster/MemberLookup_ITCase.java index 485bd42fa..08a04c3a1 100644 --- a/test/src/test/java/com/alibaba/nacos/test/core/cluster/MemberLookup_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/core/cluster/MemberLookup_ITCase.java @@ -17,7 +17,6 @@ package com.alibaba.nacos.test.core.cluster; import com.alibaba.nacos.api.exception.NacosException; -import com.alibaba.nacos.core.utils.DiskUtils; import com.alibaba.nacos.core.cluster.Member; import com.alibaba.nacos.core.cluster.ServerMemberManager; import com.alibaba.nacos.core.cluster.lookup.AddressServerMemberLookup; @@ -25,8 +24,9 @@ import com.alibaba.nacos.core.cluster.lookup.FileConfigMemberLookup; import com.alibaba.nacos.core.cluster.lookup.LookupFactory; import com.alibaba.nacos.core.cluster.MemberLookup; import com.alibaba.nacos.core.cluster.lookup.StandaloneMemberLookup; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.InetUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.DiskUtils; +import com.alibaba.nacos.sys.utils.InetUtils; import com.alibaba.nacos.test.BaseTest; import org.apache.commons.lang3.StringUtils; import org.junit.After; @@ -69,7 +69,7 @@ public class MemberLookup_ITCase extends BaseTest { DiskUtils.forceMkdir(Paths.get(path, "conf").toString()); File file = Paths.get(path, "conf", name).toFile(); DiskUtils.touch(file); - String ip = InetUtils.getSelfIp(); + String ip = InetUtils.getSelfIP(); DiskUtils.writeFile(file, (ip + ":8848," + ip + ":8847," + ip + ":8849").getBytes( StandardCharsets.UTF_8), false); diff --git a/test/src/test/java/com/alibaba/nacos/test/core/cluster/ServerMemberManager_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/core/cluster/ServerMemberManager_ITCase.java index 9579c742e..295571569 100644 --- a/test/src/test/java/com/alibaba/nacos/test/core/cluster/ServerMemberManager_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/core/cluster/ServerMemberManager_ITCase.java @@ -24,8 +24,8 @@ import com.alibaba.nacos.core.cluster.MembersChangeEvent; import com.alibaba.nacos.core.cluster.MemberUtils; import com.alibaba.nacos.core.cluster.NodeState; import com.alibaba.nacos.core.cluster.ServerMemberManager; -import com.alibaba.nacos.core.utils.ApplicationUtils; -import com.alibaba.nacos.core.utils.Constants; +import com.alibaba.nacos.sys.utils.ApplicationUtils; +import com.alibaba.nacos.sys.env.Constants; import org.junit.After; import org.junit.AfterClass; import org.junit.Assert; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/AutoDeregisterInstance_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/AutoDeregisterInstance_ITCase.java index 689822651..5053ee2c5 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/AutoDeregisterInstance_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/AutoDeregisterInstance_ITCase.java @@ -22,7 +22,7 @@ import com.alibaba.nacos.api.naming.NamingService; import com.alibaba.nacos.api.naming.pojo.Instance; import com.alibaba.nacos.client.naming.NacosNamingService; import com.alibaba.nacos.client.naming.beat.BeatInfo; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.After; import org.junit.Assert; import org.junit.Before; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/ClientBeat_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/ClientBeat_ITCase.java index 06e8adea3..6e5cefe02 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/ClientBeat_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/ClientBeat_ITCase.java @@ -21,7 +21,7 @@ import com.alibaba.nacos.Nacos; import com.alibaba.nacos.api.naming.NamingFactory; import com.alibaba.nacos.api.naming.NamingService; import com.alibaba.nacos.api.naming.pojo.Instance; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.test.base.Params; import org.apache.commons.lang3.StringUtils; import org.junit.Assert; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/DeregisterInstance_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/DeregisterInstance_ITCase.java index cb873b2c8..179c007cc 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/DeregisterInstance_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/DeregisterInstance_ITCase.java @@ -19,7 +19,7 @@ import com.alibaba.nacos.Nacos; import com.alibaba.nacos.api.naming.NamingFactory; import com.alibaba.nacos.api.naming.NamingService; import com.alibaba.nacos.api.naming.pojo.Instance; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Before; import org.junit.Test; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/MultiTenant_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/MultiTenant_ITCase.java index 418e11b9a..1d4178bf8 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/MultiTenant_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/MultiTenant_ITCase.java @@ -26,7 +26,7 @@ import com.alibaba.nacos.api.naming.listener.EventListener; import com.alibaba.nacos.api.naming.listener.NamingEvent; import com.alibaba.nacos.api.naming.pojo.Instance; import com.alibaba.nacos.api.naming.pojo.ListView; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Before; import org.junit.Test; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/NamingMaintainService_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/NamingMaintainService_ITCase.java index 0ab4df760..047d40c23 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/NamingMaintainService_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/NamingMaintainService_ITCase.java @@ -27,7 +27,7 @@ import com.alibaba.nacos.api.naming.pojo.Instance; import com.alibaba.nacos.api.naming.pojo.Service; import com.alibaba.nacos.api.selector.ExpressionSelector; import com.alibaba.nacos.api.selector.NoneSelector; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import com.alibaba.nacos.test.BaseTest; import org.junit.Assert; import org.junit.Before; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/RegisterInstance_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/RegisterInstance_ITCase.java index 0df3dbbc7..0227f3a60 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/RegisterInstance_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/RegisterInstance_ITCase.java @@ -22,7 +22,7 @@ import com.alibaba.nacos.api.naming.NamingFactory; import com.alibaba.nacos.api.naming.NamingService; import com.alibaba.nacos.api.naming.PreservedMetadataKeys; import com.alibaba.nacos.api.naming.pojo.Instance; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Before; import org.junit.Test; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/SelectInstances_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/SelectInstances_ITCase.java index a55c8afdb..4b78ef75f 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/SelectInstances_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/SelectInstances_ITCase.java @@ -22,7 +22,7 @@ import com.alibaba.nacos.api.naming.NamingService; import com.alibaba.nacos.api.naming.pojo.Instance; import com.alibaba.nacos.api.naming.pojo.ListView; import com.alibaba.nacos.api.selector.ExpressionSelector; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Before; import org.junit.Test; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/SelectOneHealthyInstance_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/SelectOneHealthyInstance_ITCase.java index f9331e46e..16104ebf0 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/SelectOneHealthyInstance_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/SelectOneHealthyInstance_ITCase.java @@ -19,7 +19,7 @@ import com.alibaba.nacos.Nacos; import com.alibaba.nacos.api.naming.NamingFactory; import com.alibaba.nacos.api.naming.NamingService; import com.alibaba.nacos.api.naming.pojo.Instance; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Before; import org.junit.Test; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/ServiceListTest_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/ServiceListTest_ITCase.java index 31963a4e7..4a58b3fed 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/ServiceListTest_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/ServiceListTest_ITCase.java @@ -25,7 +25,7 @@ import com.alibaba.nacos.api.naming.listener.NamingEvent; import com.alibaba.nacos.api.naming.pojo.Instance; import com.alibaba.nacos.api.naming.pojo.ListView; import com.alibaba.nacos.api.naming.pojo.ServiceInfo; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Before; import org.junit.Test; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/SubscribeCluster_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/SubscribeCluster_ITCase.java index ff1a42f39..86c13f809 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/SubscribeCluster_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/SubscribeCluster_ITCase.java @@ -22,7 +22,7 @@ import com.alibaba.nacos.api.naming.listener.Event; import com.alibaba.nacos.api.naming.listener.EventListener; import com.alibaba.nacos.api.naming.listener.NamingEvent; import com.alibaba.nacos.api.naming.pojo.Instance; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Before; import org.junit.Test; diff --git a/test/src/test/java/com/alibaba/nacos/test/naming/Unsubscribe_ITCase.java b/test/src/test/java/com/alibaba/nacos/test/naming/Unsubscribe_ITCase.java index 11454bb2a..a04c200d7 100644 --- a/test/src/test/java/com/alibaba/nacos/test/naming/Unsubscribe_ITCase.java +++ b/test/src/test/java/com/alibaba/nacos/test/naming/Unsubscribe_ITCase.java @@ -22,7 +22,7 @@ import com.alibaba.nacos.api.naming.listener.Event; import com.alibaba.nacos.api.naming.listener.EventListener; import com.alibaba.nacos.api.naming.listener.NamingEvent; import com.alibaba.nacos.api.naming.pojo.Instance; -import com.alibaba.nacos.core.utils.ApplicationUtils; +import com.alibaba.nacos.sys.utils.ApplicationUtils; import org.junit.Assert; import org.junit.Before; import org.junit.Test;