mirrors/APIJSON

Fork 0

TommyLemon 56378139f5

Add files via upload

2022-11-26 21:36:22 +08:00

21 KiB

Raw Blame History

功能点

一个json(事务)同时支持新增、修改、删除、查询、别名

https://github.com/Tencent/APIJSON/issues/468

使用说明

json支持多种方式定义method

第一种:

"@post","@put","@delete","@head","@get","@gets","@head","@heads"

"@post": ["Moment","Comment[]"] , 值为数组格式, 每个value = key

需要保证每个key唯一, 唯一判断标准:

key = Moment

key= Moment[]

会判断为相同key. 请通过别名区分, 别名格式: Sys_user_role:sur xxx表名:别名

{
   "@post": ["Moment","Comment:cArray[]","User:u"], // 分发到 POST 请求对应的解析处理
   "Moment": {
     // TODO 其它字段
   },
   "Comment:cArray[]": [
      {
        // TODO 其它字段
      }
   ],
   "@get": ["User"], // 分发到 GET 请求对应的解析处理
   "User:u": {
     // TODO 其它字段
   },
   "Privacy": { // 按 URL 对应的默认方法处理
     // TODO 其它字段
   }
}

对于没有显式声明操作方法的，直接用 URL(/get, /post 等) 对应的默认操作方法

第二种:

对象内定义"@method": "GET", value大写

{
	"sql@": {
    	"@method": "GET",
        "with": true,
        "from": "Sys_role",
        "Sys_role": {
          "@column": "id",
          "role_name": "角色1"
        }
    },
    "Sys_user_role:sur[]": {
    	"@method": "GET",
        "Sys_user_role": {
            "role_id{}@": "sql"
        }
    },
    "Sys_role_permission:srp[]": {
    	"@method": "GET",
        "Sys_role_permission": {
            "role_id{}@": "sql"
        }
    },
    "@explain": true
}

解析顺序

对象内 "@method"
"@post","@put","@delete"
对于没有显式声明操作方法的，直接用 URL(/get, /post 等) 对应的默认操作方法

tag自动生成规则

/**

{ "xxx:aa":{ "@tag": "" }}
生成规则:
1、@tag存在,tag=@tag
2、@tag不存在
1)、存在别名
key=对象: tag=key去除别名
key=数组: tag=key去除别名 + []
2)、不存在别名
tag=key
tag=key + [] */

建议

一个json包含不同操作方法, url method 使用 /post, /put
value为JSONArray, 建议通过"@post" 方式配置, 如果没有配置,执行 3

Request表配置说明

这只是我的配置仅供参考, 后续测试会用到:

单条新增:
POST   User_address     {"MUST":"addr","UPDATE": {"@role": "OWNER,ADMIN","childFunTest-()": "childFunTest(addr)"}, "REFUSE": "id"}
批量新增:
POST User_address[]  {"User_address[]": [{"MUST":"addr","REFUSE": "id"}], "UPDATE": {"@role": "OWNER,ADMIN","childFunTest-()": "childFunTest(addr)"}}
单条修改:
PUT User_address   {"User_address":{ "MUST":"id","REFUSE": "userId", "UPDATE": {"@role": "OWNER,ADMIN","childFunTest-()": "childFunTest(addr)"}} }
批量修改:
PUT User_address[]  {"User_address[]": [{"MUST": "id","REFUSE": "userId"}], "UPDATE": {"@role": "OWNER,ADMIN"}}
删除:
DELETE User_address   {"User_address":{ "MUST":"id{}","REFUSE": "!", "INSERT": {"@role": "OWNER,ADMIN"}} }

别名

格式:

Sys_user_role:sur xxx表名:别名

Comment:cArray[]

实现思路

当时参考了作者的示例: 注册流程. 看到绕过校验, 可以将多条json语句组装在一起, 批量执行. 于是就想如何实现一个json支持不同操作方法,并支持事物.

通过分析源码, 熟悉了校验流程、json解析执行流程、json解析生成sql语句流程、一些兼容、校验规则等

经过和作者讨论, 很感谢作者提供了相关解决方案和思路. 逐步理解了apijson的设计原理和框架结构.

一个json(事务)同时支持新增、修改、删除、查询、别名, 实现思路如下:

1、校验模块

将json解析成对象、临时变量、子查询、别名、tag等

并将method 添加到 json对象属性中.

"Sys_role": {
    	"@method": "PUT",
		"id": "6aedce0d-2a29-4fbe-aeed-0ba935ca6b41",
        "id{}@": "sql",
        "role_code": "code-subrange-4",
        "role_name": "角色-subrange-4"
    }

2、对象解析

用对象属性@method , 替换 Parser 的 method

3、事物支持

后续优化建议

1、独立定义一个url method, 通过解析不同method执行不同流程

和已有method区分开,避免歧义

2、最外层新增传参 "transaction": true 来指定开启事务目前是url put、post来控制开启事物, 以及提交的时候在 AbstractParser onCommit 判断 transactionIsolation (4 : 开启事物, 0: 非事物请求)

详细实现请参见: https://github.com/Tencent/APIJSON/issues/468

3、完善 "@Explain"

如果没有执行计划,则返回sql语句. 能够在 reponse返回值中, 看到json中执行的每条sql,方便排错

4、@version支持

定义不同场景的新增、修改、删除等执行规则. 请通过version版本区分

Request表是通过tag、method、version来保证唯一.

5、前置函数

前置函数能够将json语句, 加入到当前事物中.

例如: 像数组一样,解析成每一条语句去执行.

mysql8 with-as表达式

前提条件

1、mysql版本: 8+

2、mysql-connector-java: 8.0.31

版本支持 with-as即可

3、druid: 1.2.15

版本支持 with-as即可

4、去掉 durid wall配置

delete子查询, druid wall 拦截器报错 sql injection violation

测试案例

查询单个range ref引用

// 测试 mysql8 with as表达式
// 用户表
// 用户角色表
// 角色表
// 示例一 单个range ref引用
{
	"sql@": {
    	"@method": "GET",
        "with": true,
        "from": "Sys_role",
        "Sys_role": {
          "@column": "id",
          "role_name": "角色1"
        }
    },
    "Sys_user_role:sur[]": {
    	"@method": "GET",
        "Sys_user_role": {
            "role_id{}@": "sql"
        }
    },
    "Sys_role_permission:srp[]": {
    	"@method": "GET",
        "Sys_role_permission": {
            "role_id{}@": "sql"
        }
    },
    "@explain": true
}  

// 第二种写法
{
	"@get": ["sql@","Sys_user_role:sur[]","Sys_role_permission:srp[]"],
	"sql@": {
        "with": true,
        "from": "Sys_role",
        "Sys_role": {
          "@column": "id",
          "role_name": "角色1"
        }
    },
    "Sys_user_role:sur[]": {
        "Sys_user_role": {
            "role_id{}@": "sql"
        }
    },
    "Sys_role_permission:srp[]": {
        "Sys_role_permission": {
            "role_id{}@": "sql"
        }
    },
    "@explain": true
}

mysql8执行结果:

mysql5.7执行结果:

查询多个range ref引用

{
    "sql@": {
        "@method": "GET", 
        "with": true, 
        "from": "Sys_role", 
        "Sys_role": {
            "@column": "id", 
            "role_name": "角色1"
        }
    }, 
    "sql_user@": {
        "@method": "GET", 
        "with": true, 
        "from": "Sys_user", 
        "Sys_user": {
            "@column": "id", 
            "id": "f0894db2-6940-4d89-a5b2-4405d0ad0c8f"
        }
    }, 
    "Sys_user_role:sur[]": {
        "@method": "GET", 
        "Sys_user_role": {
            "role_id{}@": "sql", 
            "user_id{}@": "sql_user"
        }
    }, 
    "Sys_role_permission:srp[]": {
        "@method": "GET", 
        "Sys_role_permission": {
            "role_id{}@": "sql"
        }
    }, 
    "@explain": true
}

mysql8执行结果:

mysql5.7执行结果:

delete子查询

{
	"sql@": {
		"@method": "GET",
        "with": true,
        "from": "Sys_role_permission",
        "Sys_role_permission": {
          "@column": "id",
		  "role_id{}": ["e7129d5f-b07e-4996-9965-9528e370a393"]
        }
    },
    "Sys_role_permission": {
    	"@method": "DELETE",
        "id{}@": "sql"
    },
    "explan": true
}

mysql8执行sql语句:

WITH  `sql` AS (SELECT `id` FROM `housekeeping`.`Sys_role_permission` WHERE  (  (`role_id` IN ('68877ee9-4cf4-4f32-86e6-16c505ca3b21'))  ) ) DELETE FROM `housekeeping`.`Sys_role_permission` WHERE  (  (`id` IN ( SELECT * FROM `sql`) )  ) 

Plain Text

mysql5.7执行结果:

DELETE FROM `housekeeping`.`Sys_role_permission` WHERE  (  (`id` IN ( SELECT * FROM (SELECT `id` FROM `housekeeping`.`Sys_role_permission` WHERE  (  (`role_id` IN ('20d337bb-9886-455f-8dce-f1cadab0ec4f'))  ) ) AS `sql`) )  ) 



Plain Text

update子查询

{
   "sql@": {
		"@method": "GET",
        "with": true,
        "from": "Sys_role_permission",
        "Sys_role_permission": {
          "@column": "role_id",
		  "id{}": ["6aedce0d-2a29-4fbe-aeed-0ba935ca6b41"]
        }
    },
    "Sys_role": {
    	"@method": "PUT",
		"id": "6aedce0d-2a29-4fbe-aeed-0ba935ca6b41",
        "id{}@": "sql",
        "role_code": "code-subrange-4",
        "role_name": "角色-subrange-4"
    },
    "@explain": true
}

第二种写法
{
	"@get": ["sql@"],
    "sql@": {
        "with": true,
        "from": "Sys_role_permission",
        "Sys_role_permission": {
          "@column": "role_id",
		  "id{}": ["c95ef2d6-bf14-42b0-bb87-038cee8c78f1"]
        }
    },
    "@put": ["Sys_role"],
    "Sys_role": {
		"id": "0bb92d96-8ca6-469e-91e8-60308ce5b835",
        "id{}@": "sql",
        "role_code": "code-subrange-4",
        "role_name": "角色-subrange-4"
    },
    "@explain": true
}

mysql8执行sql语句:

mysql5.7执行结果:

GETS 单条子查询

会执行校验流程

http://localhost:8675/lowCodePlatform/forms/api/gets

{
	"sql@": {
        "with": true,
        "from": "Sys_user_role",
        "Sys_user_role": {
          "@column": "role_id",
		  "user_id": "4732209c-5785-4827-b532-5092f154fd94"
        }
    },
    "Sys_role[]": {
        "Sys_role": {
            "id{}@": "sql"
        },
        "page": 0,
        "count": 10,
        "query": 2
    },
    "tag":"Sys_role[]",
    "total@": "/Sys_role[]/total",
    "@explain": true
}

第二种写法
{
	"@gets": ["sql@","Sys_role[]"],
	"sql@": {
        "with": true,
        "from": "Sys_user_role",
        "Sys_user_role": {
          "@column": "role_id",
		  "user_id": "4732209c-5785-4827-b532-5092f154fd94"
        }
    },
    "Sys_role[]": {
        "Sys_role": {
            "id{}@": "sql"
        },
        "page": 0,
        "count": 10,
        "query": 2
    },
    "tag":"Sys_role[]",
    "total@": "/Sys_role[]/total",
    "@explain": true
}

Access、Request需要配置鉴权信息:

mysql8执行sql语句:

mysql5.7执行结果:

GETS多条子查询

会执行校验流程

http://localhost:8675/lowCodePlatform/forms/api/gets

{
	"sql@": {
    	"@method": "GETS",
        "with": true,
        "from": "Sys_role",
        "Sys_role": {
          "@column": "id",
          "role_name": "超级管理员"
        }
    },
    "sql_user@": {
    	"@method": "GETS",
        "with": true,
        "from": "Sys_user",
        "Sys_user": {
          "@column": "id",
          "id": "4732209c-5785-4827-b532-5092f154fd94"
        }
    },
    "Sys_user_role:sur[]": {
    	"@method": "GETS",
        "Sys_user_role": {
            "role_id{}@": "sql",
            "user_id{}@": "sql_user"
        }
    },
    "Sys_role_permission:srp[]": {
    	"@method": "GETS",
        "Sys_role_permission": {
            "role_id{}@": "sql"
        }
    },
    "@explain": true
}

mysql8执行sql语句:

mysql5.7执行结果:

head 单个子查询

普通获取数量, get/head不执行校验流程

http://localhost:8675/lowCodePlatform/forms/api/head
{
	"sql@": {
		"@method": "GET",
        "with": true,
        "from": "Sys_user_role",
        "Sys_user_role": {
          "@column": "role_id",
		  "user_id": "4732209c-5785-4827-b532-5092f154fd94"
        }
    },
    "Sys_role": {
    	"@method": "head",
        "id{}@": "sql"
    },
    "@explain": true
}

mysql8执行sql语句:

mysql5.7执行结果:

head 多个子查询

普通获取数量, get/head不执行校验流程

{
	"sql@": {
    	"@method": "GET",
        "with": true,
        "from": "Sys_role",
        "Sys_role": {
          "@column": "id",
          "role_name": "超级管理员"
        }
    },
    "sql_user@": {
    	"@method": "GET",
        "with": true,
        "from": "Sys_user",
        "Sys_user": {
          "@column": "id",
          "id": "4732209c-5785-4827-b532-5092f154fd94"
        }
    },
    "Sys_user_role": {
        "@method": "HEAD",
        "role_id{}@": "sql",
            "user_id{}@": "sql_user"
    },
    "Sys_role_permission": {
        "@method": "HEAD",
        "role_id{}@": "sql"
    },
    "@explain": true
}

mysql8执行sql语句:

mysql5.7执行结果:

heads 单个子查询

普通获取数量

会执行校验流程, Access、Request需要配置鉴权信息:

http://localhost:8675/lowCodePlatform/forms/api/heads

{
	"sql@": {
		"@method": "GET",
        "with": true,
        "from": "Sys_user_role",
        "Sys_user_role": {
          "@column": "role_id",
		  "user_id": "4732209c-5785-4827-b532-5092f154fd94"
        }
    },
    "Sys_role": {
    	"@method": "heads",
        "id{}@": "sql"
    },
    "@explain": true
}

mysql8执行sql语句:

mysql5.7执行结果:

heads 多个子查询

会执行校验流程, Access、Request需要配置鉴权信息:

普通获取数量

{
	"sql@": {
    	"@method": "GET",
        "with": true,
        "from": "Sys_role",
        "Sys_role": {
          "@column": "id",
          "role_name": "超级管理员"
        }
    },
    "sql_user@": {
    	"@method": "GET",
        "with": true,
        "from": "Sys_user",
        "Sys_user": {
          "@column": "id",
          "id": "4732209c-5785-4827-b532-5092f154fd94"
        }
    },
    "Sys_user_role": {
        "@method": "HEADS",
        "role_id{}@": "sql",
        "user_id{}@": "sql_user"
    },
    "Sys_role_permission": {
        "@method": "HEADS",
        "role_id{}@": "sql"
    },
    "@explain": true
}

mysql8执行sql语句:

mysql5.7执行结果:

delete、put 支持子查询

https://github.com/Tencent/APIJSON/issues/471

静态变量做全局处理，特殊接口用 Operation.MUST id/id{}/id{}@ 做自定义处理。

之所以默认必传，是因为安全意识不够、编码粗心大意的人太多了，所以要有一个底线保障，尽可能避免安全隐患。

1、全局配置为 PUT, DELETE 强制要求必须有 id/id{}/id{}@ 条件

AbstractVerifier.IS_UPDATE_MUST_HAVE_ID_CONDITION = true; // true: 必须有

2、细粒度控制

使用说明

// 条件删除
{
    "User:del": {
        "username": "test3"
    },
    "tag": "User",
    "explain": true
}

// 引用id{}@删除
{
	"sql@": {
		"@method": "GET",
        "with": true,
        "from": "Sys_user_role",
        "Sys_user_role": {
          "@column": "user_id",
		  "role_id{}": ["023e1880-c0d4-4e7c-ae6c-7703199c2daf"]
        }
    },
    "Sys_user:aa": {
    	"@method": "DELETE",
        "id{}@": "sql"
    },
    "explan": true
}
// 子查询条件删除
http://localhost:8675/lowCodePlatform/forms/api/delete
{
	"sql@": {
		"@method": "GET",
        "with": true,
        "from": "User",
        "User": {
          "@column": "username",
		  "username": "test-3"
        }
    },
    "User": {
        "username{}@": "sql"
    },
    "explan": true
}

第二种写法:
{
	"@get": ["sql@"],
	"sql@": {
        "with": true,
        "from": "User",
        "User": {
          "@column": "username",
		  "username": "test4"
        }
    },
    "User": {
        "username{}@": "sql"
    },
    "explan": true
}

开启id删除, 删除失败:

{
	"@get": ["sql@"],
	"sql@": {
        "with": true,
        "from": "User",
        "User": {
          "@column": "username",
		  "username": "test4"
        }
    },
    "User": {
        "username{}@": "sql"
    },
    "explan": true
}

开启id删除、id引用删除成功

{
	"sql@": {
		"@method": "GET",
        "with": true,
        "from": "Sys_user_role",
        "Sys_user_role": {
          "@column": "user_id",
		  "role_id{}": ["0bb92d96-8ca6-469e-91e8-60308ce5b835"]
        }
    },
    "Sys_user:aa": {
    	"@method": "DELETE",
        "id{}@": "sql"
    },
    "explan": true
}

PUT 子查询修改

{
   "sql@": {
		"@method": "GET",
        "with": true,
        "from": "Sys_role_permission",
        "Sys_role_permission": {
          "@column": "role_id",
		  "id{}": ["ba2634f8-0bdc-4b50-9c5e-47786b1536ef"]
        }
    },
    "Sys_role": {
    	"@method": "PUT",
        "id{}@": "sql",
        "role_code": "code-subrange-5",
        "role_name": "角色-subrange-5"
    },
    "@explain": true
}

bug修复

删除操作主表和子查询是同一张表 mysql8以下非with-as表达式会报错: "msg": "You can't specify target table 'User' for update in FROM clause",

需要调整sql语句,将子查询包一层(select * from (子查询) as xxx) DELETE FROM housekeeping.User WHERE ( (username IN (SELECT * FROM (SELECT username FROM housekeeping.User WHERE ( (username = 'test1') )) as a) ) )

must、refuses判断、delete、PUT支持 ref

{
	"sql@": {
		"@method": "GET",
        "with": true,
        "from": "Sys_role_permission",
        "Sys_role_permission": {
          "@column": "id",
		  "role_id{}": ["94f79f0b-331b-4cc5-bfc0-ebfc47d00f13"]
        }
    },
    "Sys_role_permission": {
    	"@method": "DELETE",
        "id{}@": "sql"
    },
    "explan": true
}

21 KiB Raw Blame History

功能点

一个json(事务)同时支持新增、修改、删除、查询、别名

使用说明

解析顺序

tag自动生成规则

建议

Request表 配置说明

别名

实现思路

后续优化建议

mysql8 with-as表达式

前提条件

测试案例

查询单个range ref引用

查询多个range ref引用

delete子查询

update子查询

GETS 单条子查询

GETS多条子查询

head 单个子查询

head 多个子查询

heads 单个子查询

heads 多个子查询

delete、put 支持子查询

使用说明

bug修复

must、refuses判断、delete、PUT支持 ref

21 KiB

Raw Blame History

Request表配置说明