CN112580098A - Business data comparison method and system - Google Patents
Business data comparison method and system Download PDFInfo
- Publication number
- CN112580098A CN112580098A CN202011536731.XA CN202011536731A CN112580098A CN 112580098 A CN112580098 A CN 112580098A CN 202011536731 A CN202011536731 A CN 202011536731A CN 112580098 A CN112580098 A CN 112580098A
- Authority
- CN
- China
- Prior art keywords
- service
- service data
- data
- processed
- buffer queue
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/62—Protecting access to data via a platform, e.g. using keys or access control rules
- G06F21/6218—Protecting access to data via a platform, e.g. using keys or access control rules to a system of files or objects, e.g. local or distributed file system or database
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24552—Database cache management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioethics (AREA)
- Software Systems (AREA)
- Computer Security & Cryptography (AREA)
- Computer Hardware Design (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a business data comparison method and a system, wherein the method comprises the following steps: step S1: receiving service data; step S2: comparing the service data to be processed in the temporary cache with the service data in the cache queue; step S3: storing the service data to be processed into a buffer queue, and continuing to process the next service data; the method specifically comprises the following steps: taking out the service to be processed from the temporary buffer and storing the service to be processed into a specific position in a buffer queue; step S4: merging the service data to be processed and the same service data in the buffer queue, and continuing to process the next service data; the invention carries out the preliminary processing of the service data by comparing the service data and simultaneously adjusts and marks the service data in the buffer queue, thereby leading the buffer queue to simultaneously meet the requirements of assignment and buffering, and greatly improving the comparison efficiency under the guidance of historical data analysis. By presuming the way of not comparing, the execution efficiency has been improved greatly.
Description
[ technical field ] A method for producing a semiconductor device
The invention belongs to the technical field of data processing, and particularly relates to a service data comparison method and system.
[ background of the invention ]
The safety, reliability and efficiency of the service data are extremely important for guaranteeing the normal processing of the service system data and the output of reliable results. Particularly, with the increasing development of internet finance, service organizations usually build their own business data systems to process business data requests of users, and how to ensure the security of the business data, the processing efficiency of the business data, and how to avoid the pressure caused by business data storage is a problem that many service enterprise users pay more and more attention to at present. On one hand, in the prior art, when business data is processed, stored and calculated, only data identification comparison is often performed, and the comparison dimension is also one-dimensional comparison. On the other hand, because the repetition of the service type and the increase of the number of the user terminals can bring a large amount of repeated data, and the repeated data can not be overcome through communication control, so that repeated comparison, storage and calculation of data can be caused, and the efficiency of service provision is greatly reduced, aiming at the problems, the invention (1) performs primary processing on the service data through service data comparison, and simultaneously adjusts and marks the service data in the buffer queue, so that the buffer queue can simultaneously meet the requirements of assignment and buffering; (2) the number range based on size judgment is adopted for determination to obtain the judgment whether the subsequent multi-dimensional comparison is needed; (3) selecting a comparison mode of the step according to a privacy mode of the service, providing an alignment comparison mode, scientifically sampling, providing an alignment method for determining the micro distance, overcoming the possible deviation problem through the micro distance determination, and greatly improving the comparison efficiency under the guidance of historical data analysis; (4) a mode of presuming no comparison is provided, and the execution efficiency is greatly improved.
[ summary of the invention ]
In order to solve the above problems in the prior art, the present invention provides a method and system for comparing service data,
step S1: receiving service data;
step S2: comparing the service data to be processed in the temporary cache with the service data in the cache queue;
step S21: rapidly comparing the size of the service data to be processed with the sizes of all the service data in the buffer queue to determine whether the service data with the same size exists, and if so, continuing to perform the next step; otherwise, go to step S3;
step S22: selecting a corresponding rough comparison mode according to the secrecy mode of the service data to be processed;
step S23: determining whether business data which can be aligned with the business data to be processed exist, if so, entering the next step, otherwise, entering the step S3;
step S24: comparing the contents of the service data to be processed and the service data in the buffer queue to determine whether the service data are the same;
step S3: storing the service data to be processed into a buffer queue, and continuing to process the next service data; the method specifically comprises the following steps: taking out the service to be processed from the temporary buffer and storing the service to be processed into a specific position in a buffer queue;
step S4: and merging the service data to be processed and the same service data in the buffer queue, and continuing to process the next service data.
Furthermore, the number of the service terminals is multiple.
Further, the business data is multi-source business data.
Further, the service data are of the same type or different types.
Furthermore, after the sizes of the service data in the buffer queue are sorted, the size of the service data to be processed and the sorted size are quickly searched to determine whether the service data with the same size exist.
Further, comparing the size of the service to be processed with the size of the service data in the buffer queue stored in a binary tree form to determine the number of the service data which is approximately the same as the size of the service to be processed; when the number is larger than the number threshold value, continuing to perform the next step; otherwise, the process proceeds to step S3.
Further, approximately the same means that the difference of the size values is within a preset range; the quantity threshold is a preset value.
A business data comparison system comprising: the system comprises a service terminal, a service processing server and a co-processing server;
the service terminal is used for submitting the service data to the service processing server;
the service processing server is used for receiving the service data, comparing the service data, assigning the service, completing the service request and returning the completed processing result to the service terminal; the service processing server comprises a buffer queue, a service assignment unit and a service processing unit, wherein the buffer queue is positioned in front of the service assignment unit and is used for comparing service data before service assignment; the service processing unit is used for processing the service data assigned by the service assignment unit; the storage device is used for storing the service data and the processing related data thereof;
the service assigning unit is used for acquiring a service data from the head of the buffer queue and assigning the service data to the service processing unit or the co-processing server for execution;
the service processing unit is used for processing the service data and returning a processing result to the service terminal;
the one or more coprocessing servers are used for processing the service assigned by the service assignment unit; returning the processing result to the service terminal; preferably: the co-processing server is a processing resource independent of the business processing server; preferably: the coprocessing server is a cloud server.
Further, wherein: the buffer queue is used for receiving the service data.
Further, the preliminary processing of the service data is performed through service data comparison, and the service data in the buffer queue is adjusted and marked in the comparison process, so that the buffer queue can receive the service data for buffering and is also used for the assignment of the service assignment unit.
The beneficial effects of the invention include: (1) the primary processing of the service data is carried out through the service data comparison, and the service data in the buffer queue is adjusted and marked at the same time, so that the buffer queue can meet the requirements of assignment and buffering at the same time; (2) the number range based on size judgment is adopted for determination to obtain the judgment whether the subsequent multi-dimensional comparison is needed; (3) selecting a comparison mode of the step according to a privacy mode of the service, providing an alignment comparison mode, scientifically sampling, providing an alignment method for determining the micro distance, overcoming the possible deviation problem through the micro distance determination, and greatly improving the comparison efficiency under the guidance of historical data analysis; (4) a mode of presuming no comparison is provided, and the execution efficiency is greatly improved.
[ description of the drawings ]
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, and are not to be considered limiting of the invention, in which:
fig. 1 is a schematic diagram of a service data comparison method according to the present invention.
[ detailed description ] embodiments
The present invention will now be described in detail with reference to the drawings and specific embodiments, wherein the exemplary embodiments and descriptions are provided only for the purpose of illustrating the present invention and are not to be construed as limiting the present invention.
A service data comparison method and system applied in the present invention will be described in detail below.
The invention relates to a service data comparison system, which comprises: the system comprises a service terminal, a service processing server and a co-processing server;
the service terminals are used for submitting service data to the service processing server;
the service processing server is used for receiving the service data, comparing the service data, assigning the service, completing the service request and returning the completed processing result to the service terminal; the service processing server comprises a buffer queue, a service assignment unit and a service processing unit, wherein the buffer queue is positioned in front of the service assignment unit and is used for comparing service data before service assignment; the service processing unit is used for processing the service data assigned by the service assignment unit; the storage device is used for storing the service data and the processing related data thereof;
wherein: the buffer queue is used for receiving service data; in the prior art, a buffer queue and an assignment queue adopted by an assignment unit are often arranged separately, and each assignment unit pair uses a service processing unit or a co-processing server, so that one important reason for the arrangement is that scheduling strategies adopted by assignment and buffering are different. The invention combines the two queues into the same queue, performs the primary processing of the service data through the service data comparison, and simultaneously adjusts and marks the service data in the buffer queue, thereby enabling the buffer queue to simultaneously meet the requirements of assignment and buffering. The service data is for example: a series of internet service services such as payment service, bicycle service, purchase service, browsing service, etc.;
the service assigning unit is used for acquiring a service data from the head of the buffer queue and assigning the service data to the service processing unit or the co-processing server for execution;
the service processing unit is used for processing the service data and returning a processing result to the service terminal;
the one or more coprocessing servers are used for processing the service assigned by the service assignment unit; returning the processing result to the service terminal; preferably: the co-processing server is a processing resource independent of the business processing server; preferably: the coprocessing server is a cloud server;
as shown in fig. 1, the method comprises the following steps:
step S1: receiving service data; specifically, the method comprises the following steps: the service terminal sends out service data, and the service processing server receives the service data and puts the service data into a temporary cache; wherein: the temporary buffer is a storage space different from the buffer queue;
preferably: the number of the service terminals is multiple, and the service data is multi-source service data;
preferably: the service data are the same or different types;
step S2: comparing the service data to be processed in the temporary cache with the service data in the cache queue; the method specifically comprises the following steps:
step S21: rapidly comparing the size of the service data to be processed with the sizes of all the service data in the buffer queue to determine whether the service data with the same size exists, and if so, continuing to perform the next step; otherwise, go to step S3;
preferably: after the sizes of the service data in the buffer queue are sorted, the sizes of the service data to be processed and the sorted sizes are quickly searched to determine whether the service data with the same size exist;
in order to match with subsequent multi-dimensional judgment, even the conclusion whether the sizes are the same does not need to be obtained accurately, the business data have detail difference, actually have the same content and are possible repeated business data, the invention provides that the number range based on the size judgment is determined to obtain the judgment whether the subsequent multi-dimensional comparison is needed; thus, the following can be substituted: comparing the size of the service to be processed with the size of the service data in the buffer queue stored in a binary tree form to determine the number of the service data which is approximately the same as the size of the service to be processed; when the number is larger than the number threshold value, continuing to perform the next step; otherwise, go to step S3; wherein: approximately the same means that the difference of the size values is within a preset range; the quantity threshold value is a preset value;
preferably: the service data with the same size or approximately the same size in the buffer queue is taken as target service data;
step S22: selecting a corresponding rough comparison mode according to the secrecy mode of the service data to be processed; specifically, the method comprises the following steps: when the service data to be processed is encrypted, step S221 is performed; otherwise, go to step S222; in the prior art, when comparing service data, the safety problem of the data itself is often not considered, or new calculation cost possibly brought by encryption and decryption itself is often not considered, and actually, unpredictable safety implications may be brought by decryption before the service itself is not processed too early;
preferably: the service data with the same rough comparison is used as target service data;
step S221: calculating a first abstract value to be compared of the service data to be processed, comparing the first abstract value to be compared with the first abstract value of each service data in the buffer queue to determine whether the service data with the same first abstract value exists, and if so, continuing to perform the next step; otherwise, go to step S3;
preferably: when the business data with the same first abstract value exist, the next step is continued only if the same quantity exceeds a certain quantity;
preferably: the abstract value is obtained by a common abstract value calculation method;
preferably: calculating the first abstract value by adopting a special calculating unit;
step S222: determining key characteristics of service data to be processed, acquiring corresponding key service data based on the key characteristics, comparing the key service data with the key service data of each service data in a buffer queue to determine whether the service data with the same key service data exists, and if so, continuing to perform the next step; otherwise, go to step S3;
preferably: decrypting the service data before acquiring the key service data; preferably: decrypting part of the service data according to the comparison requirement, namely decrypting only the key service data part;
the determining of the key characteristics of the service data to be processed specifically includes: for each feature with a feature value in the service data, obtaining a variation range DVari corresponding to each feature according to a relation Q () between the feature CHi and a service data variation range, and taking the feature with the maximum variation range DVari or exceeding a range threshold as a key feature; wherein: the range threshold is a preset value;
preferably: acquiring historical service data, and analyzing the relation between the change condition of the service data and each characteristic change; thereby converting the multi-dimensional feature problem into a one-dimensional problem. The service data is considered to be changed when the content of the service data is changed; thus, the relation Q (), between each characteristic CHi and DVar in the service data variation range, is obtained; DVar ═ q (chi); the range here may be a relative address range; or a range of sizes, etc.;
the acquiring of the corresponding key service data based on the key features specifically includes: acquiring a corresponding comparison table according to the type of service data, acquiring the position of key service data corresponding to key features based on the key features, and acquiring the key service data according to the position; because the organization mode of each type of service type is corresponding to the type, and therefore a separate setting and type-related comparison table is needed, the invention firstly directly locates the data which is most likely to change and has meaningful change through the key service data, and the data is obtained through historical data analysis, the analysis is the influence size, and the influence size and the substantial position of the data have no relation; then, starting from the logical relationship through the organization relationship of the business data, performing comparison and collection on the part of key data; for example: after determining the name of the key feature user, acquiring corresponding service data, taking three names as key data, and directly acquiring the partial data to compare to determine whether the whole service data is the same; thus greatly reducing the comparison time; the whole service data block may be different from the part of data, so that different data are directly positioned;
step S23: determining whether business data which can be aligned with the business data to be processed exist, if so, entering the next step, otherwise, entering the step S3; preferably: the service data which can be aligned is used as target service data; direct byte-by-byte comparison obviously has too large cost, the invention provides an alignment comparison mode, can scientifically sample, provides an alignment method for determining micro distance, overcomes the possible deviation problem through micro distance determination, and greatly improves the comparison efficiency under the guidance of historical data analysis;
the determining whether there is service data that can be aligned with the service data to be processed includes:
step S231: determining a random number;
preferably: if the random number is larger than the length of the service data to be processed, performing modulo operation on the random number based on the length of the service data to be processed to obtain a remainder as a random number value;
step S232: acquiring the abstract value of a preset length data block of the to-be-processed service data from the random numerical value position, and acquiring the abstract value of the preset length data block at the corresponding position of each service data in the buffer queue; if the two are the same, continuing to add the span value to the random number position to obtain a next position, and continuing to compare the next position; if the positions are different, carrying out position alignment, and comparing based on the aligned positions; if the abstract values of the number of the preset positions are the same, the service data and the service data to be processed can be aligned; if the service data which can be aligned or the alignment quantity of which exceeds the alignment preset quantity exists, continuing to perform the next step; otherwise, go to step S3;
preferably: the span value (for example: 256 bytes) and the alignment preset number (for example: 2) are preset values;
the position alignment is specifically as follows: calculating the address range XZ, XZ ═ G (f (tosv), t, TM); selecting an address change value xi from the address range, determining the micro distance, and if the micro distance is determined, determining that the mobile terminal can be aligned after moving; taking the address change value x as a movement length x; moving the x unit addresses forwards or backwards by taking the corresponding positions as centers, and taking the moved address positions as aligned positions; wherein: ToSv is the length of the service to be processed, and f (ToSv) is the size range corresponding to the length; t is the current time, TM is the type of the service terminal sending the service to be processed (the type can be a user label, a service type, etc.); g () is the most common address range determined according to the size range of the service data to be processed, the current time and the terminal type; can be obtained through historical data analysis;
the micro-distance determination specifically comprises the following steps: taking the corresponding position as a center, moving xi unit addresses forwards or backwards to obtain a micro-distance determination address, and considering that the micro-distance is determined if preset bit data starting from the random numerical value position of the service data to be processed is the same as the preset bit data starting from the distance determination position of the service data in the buffer queue; if not, continuing to select the address change value xi; the preset bit data amount is small and can be one byte or 32 bits and the like;
preferably: the selected address variation value xi can be selected randomly; or when address offset is carried out, directly taking the last value of the offset address register as the randomly selected value; therefore, when a large number of repetitions are processed, the determination can be quickly carried out, and the efficiency is greatly improved; different from the mode of directly comparing original positions in the prior art, the difference of the service data surface caused by the difference of a communication system, an operating system and the like is avoided through position alignment; during alignment, a micro-distance determination and alignment matching mode is carried out, and the value of x can be accurately positioned by combining big data guidance, so that the alignment fails;
preferably: the number of the preset positions is 2; the preset number of alignments is a preset value, for example: 3 times;
the obtaining of the digest value of the preset length data block at the corresponding position specifically includes: calculating a bit-by-bit addition value of a data block with a preset length, and taking the addition as a digest value; the mode of calculating the abstract value is selected as a relatively simple calculating mode;
step S24: comparing the contents of the service data to be processed and the service data in the buffer queue to determine whether the service data are the same; where content comparison is bit-by-bit comparison, rather than partial content-based comparison; the service data which are completely the same can be determined through content comparison;
preferably: only comparing the service data to be processed with the target service data in the buffer queue;
in the prior art, the comparison of service data is usually direct bit-by-bit comparison, and the comparison of the invention is supported by the steps, so that a mode of presuming no comparison is provided, and through historical data analysis, all the business data which are presumed to be counted and compared to the step are not the same in rate, namely the failure rate of the steps, if the failure rate is acceptable, detailed contents, namely bit-by-bit comparison is not carried out, and the loss of failure is left to the client terminal or the server;
preferably: recording the number of times of entering the step S24 and determining that the same service data as the service to be processed does not exist in the buffer queue through content comparison and the occurrence ratio thereof, if the ratio is smaller than a preset ratio value, performing no content comparison when entering the step next time, directly determining that the same service data exists and entering the step S4;
step S3: storing the service data to be processed into a buffer queue, and continuing to process the next service data; the method specifically comprises the following steps: taking out the service to be processed from the temporary buffer and storing the service to be processed into a specific position in a buffer queue;
preferably: the specific position is the tail of the buffer queue;
preferably: storing the service to be processed in a buffer queue segment corresponding to the priority in a buffer queue based on the priority of the service to be processed;
step S4: merging the service data to be processed and the same service data in the buffer queue, and continuing to process the next service data; merging here means that the service data to be processed is not stored in the buffer queue any more, marking the service data in the buffer queue, which is the same as the service data to be processed, and marking the service terminal which sends the service data request; after the service data is processed, feeding back a processing result to a service terminal related to the mark according to the mark content;
through service data comparison, the processing amount of the service data is greatly reduced, meanwhile, the storage space is also reduced, and the processing efficiency is improved;
preferably: storing the tag in a service data header;
preferably: the method further comprises step S5;
step S5: periodically counting the execution times of each step in the step S2, and determining a step template corresponding to the execution time distribution based on the execution time distribution, wherein the step template comprises the sub-steps required in the step S2 and the arrangement sequence thereof;
in the prior art, comparison processing from beginning to end is often performed through an execution flow, or complete adaptive comparison is performed, so that customization of each step is difficult, the invention provides a semi-adaptive comparison processing method, semi-customization of a processing flow can be performed aiming at an external environment with complex service data change conditions, and dynamic adjustment is performed according to a comparison result, for example: when the repeated service data is more, the processing complexity is increased by carrying out gradual comparison, the previous comparison steps of quick comparison, abstract value comparison, alignment and the like are directly skipped, and the final comparison determination is directly carried out, so that the execution efficiency can be greatly improved;
as will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, as for the apparatus embodiment, since it is substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Those skilled in the art will appreciate that all or part of the steps in the above method embodiments may be implemented by a program to instruct relevant hardware to perform the steps, and the program may be stored in a computer-readable storage medium, which is referred to herein as a storage medium, such as: ROM/RAM, magnetic disk, optical disk, etc.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (10)
1. A method for comparing service data, the method comprising:
step S1: receiving service data;
step S2: comparing the service data to be processed in the temporary cache with the service data in the cache queue;
step S21: rapidly comparing the size of the service data to be processed with the sizes of all the service data in the buffer queue to determine whether the service data with the same size exists, and if so, continuing to perform the next step; otherwise, go to step S3;
step S22: selecting a corresponding rough comparison mode according to the secrecy mode of the service data to be processed;
step S23: determining whether business data which can be aligned with the business data to be processed exist, if so, entering the next step, otherwise, entering the step S3;
step S24: comparing the contents of the service data to be processed and the service data in the buffer queue to determine whether the service data are the same;
step S3: storing the service data to be processed into a buffer queue, and continuing to process the next service data; the method specifically comprises the following steps: taking out the service to be processed from the temporary buffer and storing the service to be processed into a specific position in a buffer queue;
step S4: and merging the service data to be processed and the same service data in the buffer queue, and continuing to process the next service data.
2. The method of claim 1, wherein the number of the service terminals is plural.
3. The business data comparison method of claim 2, wherein the business data is multi-source business data.
4. The method of claim 3, wherein the service data are of the same or different types.
5. The service data comparison method according to claim 4, wherein after sorting the sizes of the service data in the buffer queue, the size of the service data to be processed and the sorted sizes are quickly searched to determine whether there is service data of the same size.
6. The traffic data comparison method according to claim 5, wherein the size of the traffic data to be processed is compared with the size of the traffic data in the buffer queue stored in a binary tree form to determine the amount of the traffic data substantially identical to the size of the traffic data to be processed; when the number is larger than the number threshold value, continuing to perform the next step; otherwise, the process proceeds to step S3.
7. The method according to claim 6, wherein substantially the same means that the difference of the size values is within a predetermined range; the quantity threshold is a preset value.
8. A service data comparison system based on the service data comparison method of any one of claims 1 to 7, comprising: the system comprises a service terminal, a service processing server and a co-processing server;
the service terminal is used for submitting the service data to the service processing server;
the service processing server is used for receiving the service data, comparing the service data, assigning the service, completing the service request and returning the completed processing result to the service terminal; the service processing server comprises a buffer queue, a service assignment unit and a service processing unit, wherein the buffer queue is positioned in front of the service assignment unit and is used for comparing service data before service assignment; the service processing unit is used for processing the service data assigned by the service assignment unit; the storage device is used for storing the service data and the processing related data thereof;
the service assigning unit is used for acquiring a service data from the head of the buffer queue and assigning the service data to the service processing unit or the co-processing server for execution;
the service processing unit is used for processing the service data and returning a processing result to the service terminal;
the one or more coprocessing servers are used for processing the service assigned by the service assignment unit; returning the processing result to the service terminal; preferably: the co-processing server is a processing resource independent of the business processing server; preferably: the coprocessing server is a cloud server.
9. The traffic data comparison system of claim 8, wherein: the buffer queue is used for receiving the service data.
10. The traffic data comparison system according to claim 9, wherein the preliminary processing of the traffic data is performed by traffic data comparison, and the traffic data in the buffer queue is adjusted and marked during the comparison, so that the buffer queue can receive the traffic data for buffering and also for the traffic assignment unit to assign.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536731.XA CN112580098A (en) | 2020-12-23 | 2020-12-23 | Business data comparison method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536731.XA CN112580098A (en) | 2020-12-23 | 2020-12-23 | Business data comparison method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112580098A true CN112580098A (en) | 2021-03-30 |
Family
ID=75139438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011536731.XA Pending CN112580098A (en) | 2020-12-23 | 2020-12-23 | Business data comparison method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112580098A (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104021034A (en) * | 2014-06-24 | 2014-09-03 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN104734992A (en) * | 2015-03-30 | 2015-06-24 | 华为技术有限公司 | Service data transmission method and device |
| CN107528825A (en) * | 2017-07-06 | 2017-12-29 | 努比亚技术有限公司 | A kind of resource downloading method, terminal and computer-readable recording medium |
| US20180324277A1 (en) * | 2017-05-03 | 2018-11-08 | International Business Machines Corporation | System and method for message queue configuration in a network |
| CN110555744A (en) * | 2018-05-31 | 2019-12-10 | 阿里巴巴集团控股有限公司 | Service data processing method and system |
| CN110661731A (en) * | 2019-09-26 | 2020-01-07 | 光大兴陇信托有限责任公司 | Message processing method and device |
| US20200027551A1 (en) * | 2018-07-17 | 2020-01-23 | Icu Medical, Inc. | Maintaining clinical messaging during network instability |
| CN111061762A (en) * | 2019-11-08 | 2020-04-24 | 京东数字科技控股有限公司 | Distributed task processing method, related device, system and storage medium |
| CN111107003A (en) * | 2019-12-31 | 2020-05-05 | 光大兴陇信托有限责任公司 | Intelligent routing method |
| CN111552696A (en) * | 2020-03-16 | 2020-08-18 | 深圳壹账通智能科技有限公司 | Data processing method and device based on big data, computer equipment and medium |
| CN111767143A (en) * | 2020-06-24 | 2020-10-13 | 中国工商银行股份有限公司 | Transaction data processing method, device, equipment and system |
| CN112104572A (en) * | 2020-09-11 | 2020-12-18 | 北京天融信网络安全技术有限公司 | Data processing method and device, electronic equipment and storage medium |
-
2020
- 2020-12-23 CN CN202011536731.XA patent/CN112580098A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104021034A (en) * | 2014-06-24 | 2014-09-03 | 北京奇艺世纪科技有限公司 | Task processing method and system |
| CN104734992A (en) * | 2015-03-30 | 2015-06-24 | 华为技术有限公司 | Service data transmission method and device |
| US20180324277A1 (en) * | 2017-05-03 | 2018-11-08 | International Business Machines Corporation | System and method for message queue configuration in a network |
| CN107528825A (en) * | 2017-07-06 | 2017-12-29 | 努比亚技术有限公司 | A kind of resource downloading method, terminal and computer-readable recording medium |
| CN110555744A (en) * | 2018-05-31 | 2019-12-10 | 阿里巴巴集团控股有限公司 | Service data processing method and system |
| US20200027551A1 (en) * | 2018-07-17 | 2020-01-23 | Icu Medical, Inc. | Maintaining clinical messaging during network instability |
| CN110661731A (en) * | 2019-09-26 | 2020-01-07 | 光大兴陇信托有限责任公司 | Message processing method and device |
| CN111061762A (en) * | 2019-11-08 | 2020-04-24 | 京东数字科技控股有限公司 | Distributed task processing method, related device, system and storage medium |
| CN111107003A (en) * | 2019-12-31 | 2020-05-05 | 光大兴陇信托有限责任公司 | Intelligent routing method |
| CN111552696A (en) * | 2020-03-16 | 2020-08-18 | 深圳壹账通智能科技有限公司 | Data processing method and device based on big data, computer equipment and medium |
| CN111767143A (en) * | 2020-06-24 | 2020-10-13 | 中国工商银行股份有限公司 | Transaction data processing method, device, equipment and system |
| CN112104572A (en) * | 2020-09-11 | 2020-12-18 | 北京天融信网络安全技术有限公司 | Data processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111814045B (en) | Data query method, device, electronic equipment and storage medium | |
| WO2022151590A1 (en) | Method, apparatus and device for performing sampling inspection on data, and storage medium | |
| CN111756522A (en) | Data processing method and system | |
| US7318160B2 (en) | Cryptographic key setup in queued cryptographic systems | |
| CN110321480B (en) | Recommendation information pushing method and device, computer equipment and storage medium | |
| CN109409079B (en) | Weak password verification method and device | |
| CN112149068A (en) | Access-based authorization verification method, information generation method and device, and server | |
| CN112235104B (en) | A data encryption transmission method, system, terminal and storage medium | |
| CN108399175A (en) | A kind of storage of data, querying method and its device | |
| CN114338548A (en) | Message distribution method, device, network equipment and computer readable storage medium | |
| CN114579602A (en) | Data retrieval method, device, storage medium and equipment based on encrypted data | |
| US8700918B2 (en) | Data masking | |
| CN112214270B (en) | A page redrawing method, device, equipment and storage medium | |
| CN113411364B (en) | Resource acquisition method and device and server | |
| CN112580098A (en) | Business data comparison method and system | |
| WO2020244069A1 (en) | Product publishing method and apparatus, computer equipment, and storage medium | |
| JPWO2021211435A5 (en) | ||
| CN116737417A (en) | Data synchronization method, device, equipment and storage medium | |
| WO2023178792A1 (en) | Ciphertext data storage method and apparatus, and device and storage medium | |
| CN115879157A (en) | Data security search method and device, equipment, medium and product thereof | |
| CN113918997A (en) | Data management system, and control method and device for data encryption and decryption | |
| CN112685706A (en) | Request authentication method and related equipment | |
| CN113691621A (en) | Block chain data uplink method | |
| CN111783157A (en) | Credible two-dimensional code electronic certificate processing method and system | |
| CN118733598B (en) | Certificate storage management method, system, terminal and medium for digital certificates |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| AD01 | Patent right deemed abandoned |
Effective date of abandoning: 20250509 |
|
| AD01 | Patent right deemed abandoned |