Skip to content

Commit 3215a67

Browse files
author
knielsen@ymer.(none)
committed
WL#2223: NdbRecord.
Mostly added documentation of protocols, interfaces, and implementation of scans in the NDB API. A few minor fixes discovered on the way.
1 parent 877615b commit 3215a67

22 files changed

+551
-100
lines changed

‎storage/ndb/include/kernel/signaldata/AttrInfo.hpp‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,4 +52,36 @@ class AttrInfo {
5252
Uint32 attrData[DataLength];
5353
};
5454

55+
/*
56+
A train of ATTRINFO signals is used to specify attributes to read or
57+
attributes and values to insert/update in TCKEYREQ, and to specify
58+
attributes to read in SCAN_TABREQ.
59+
60+
The ATTRINFO signal train defines a stream of attribute info words. (Note
61+
that for TCKEYREQ, the first five words are stored inside the TCKEYREQ
62+
signal. For SCAN_TABREQ, all attribute info words are sent in ATTRINFO
63+
signals).
64+
65+
For SCAN_TABREQ, the attribute information can have up to five sections. The
66+
initial five words of the stream defines the length of the sections,
67+
followed by the words of each section in sequence.
68+
69+
The sections are:
70+
1. Attributes to read before starting any interpreted program.
71+
2. Interpreted program.
72+
3. Attributes to update after running interpreted program.
73+
4. Attributes to read after interpreted program.
74+
5. Subroutine data.
75+
76+
The formats of sections that specify attributes to read or update is a
77+
sequence of entries, each (1+N) words:
78+
1 word specifying the AttributeHeader (attribute id in upper 16 bits, and
79+
size in bytes of data in lower 16 bits).
80+
N words of data (N = (data byte length+3)>>2).
81+
For specifying attributes to read, the data length is always zero.
82+
For an index range scan of a table using an ordered index, the attribute IDs
83+
refer to columns in the underlying table, not to columns being indexed, so
84+
all attributes in the underlying table being indexed are accessible.
85+
*/
86+
5587
#endif

‎storage/ndb/include/kernel/signaldata/KeyInfo.hpp‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,4 +45,27 @@ class KeyInfo {
4545
Uint32 keyData[DataLength];
4646
};
4747

48+
/*
49+
The KEYINFO signal is used to send a stream of data defining keys for
50+
primary key operations (TCKEYREQ) or ordered index scan bounds (SCAN_TABREQ).
51+
52+
For TCKEYREQ, the first 8 words of the KEYINFO stream are actually stored
53+
inside the TCKEYREQ signal, so for shorter keys, no KEYINFO signals are
54+
needed. Otherwise as many consecutive KEYINFO signals as needed are sent with
55+
max KeyInfo::Datalength words of data in each.
56+
57+
For scan bounds for ordered indexes, the data sent consists of a sequence of
58+
entries, each (2+N) words:
59+
1 word of bound type (0:<= 1:< 2:>= 3:> 4:==)
60+
1 word of AttributeHeader (containing attribute Id and byte length)
61+
N words of attribute data (N = (length+3)>>2).
62+
Additionally, it is possible to send multiple range bounds in a single
63+
SCAN_TABREQ and associated ATTRINFO stream (using
64+
NdbIndexScanOperation::end_of_bound(RANGE_NO) ). In this case, the first word
65+
of each range bound contains additional information: bits 16-31 holds the
66+
length of this bound, in words of ATTRINFO data, and bits 4-11 holds a
67+
number RANGE_NO specified by the application that can be read back from the
68+
RANGE_NO pseudo-column.
69+
70+
*/
4871
#endif

‎storage/ndb/include/kernel/signaldata/ScanFrag.hpp‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,18 @@ class ScanFragReq {
7676
static void setLcpScanFlag(Uint32 & requestInfo, Uint32 val);
7777
};
7878

79+
/*
80+
The KEYINFO20 signal is sent from LQH to API for each row in a scan when the
81+
ScanTabReq::getKeyinfoFlag() is set in requestInfo in the SCAN_TABREQ signal.
82+
83+
The '20' in the signal name refers to the number of keyInfo data words in
84+
the signal, which is actually a bit misleading since now it is sent as a
85+
single long signal if the keyinfo has more than 20 words.
86+
87+
The information in this signal is used in the NDB API to request the take
88+
over of a lock from the scan with a TCKEYREQ, using the primary key info
89+
sent as data and the scanInfo_Node word to identify the lock.
90+
*/
7991
class KeyInfo20 {
8092
/**
8193
* Sender(s)
@@ -100,10 +112,23 @@ class KeyInfo20 {
100112
public:
101113
Uint32 clientOpPtr;
102114
Uint32 keyLen;
115+
/*
116+
The scanInfo_Node word contains the information needed to identify the
117+
row and lock to take over in the TCKEYREQ signal. It has two parts:
118+
1. ScanInfo Lower 20 bits
119+
2. ScanFragment Upper 14 bits
120+
*/
103121
Uint32 scanInfo_Node;
104122
Uint32 transId1;
105123
Uint32 transId2;
106124
Uint32 keyData[DataLength];
125+
/*
126+
Note that if the key info data does not fit within the maximum of 20
127+
in-signal words, the entire key info is instead sent in long signal
128+
section 0.
129+
The data here is a word string suitable for sending as KEYINFO in
130+
the TCKEYREQ signal.
131+
*/
107132
};
108133

109134
class ScanFragConf {

‎storage/ndb/include/kernel/signaldata/ScanTab.hpp‎

Lines changed: 43 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,10 @@ class ScanTabReq {
5858
UintR apiConnectPtr; // DATA 0
5959
UintR attrLenKeyLen; // DATA 1
6060
UintR requestInfo; // DATA 2
61+
/*
62+
Table ID. Note that for a range scan of a table using an ordered index,
63+
tableID is the ID of the index, not of the underlying table.
64+
*/
6165
UintR tableId; // DATA 3
6266
UintR tableSchemaVersion; // DATA 4
6367
UintR storedProcId; // DATA 5
@@ -111,7 +115,11 @@ class ScanTabReq {
111115
l = Lock mode - 1 Bit 8
112116
h = Hold lock mode - 1 Bit 10
113117
c = Read Committed - 1 Bit 11
114-
k = Keyinfo - 1 Bit 12
118+
k = Keyinfo - 1 Bit 12 If set, LQH will send back a KEYINFO20
119+
signal for each scanned row,
120+
containing information needed to
121+
identify the row for subsequent
122+
TCKEYREQ signal(s).
115123
t = Tup scan - 1 Bit 13
116124
z = Descending (TUX) - 1 Bit 14
117125
x = Range Scan (TUX) - 1 Bit 15
@@ -347,7 +355,16 @@ class ScanTabConf {
347355

348356
struct OpData {
349357
Uint32 apiPtrI;
358+
/*
359+
tcPtrI is the scan fragment record pointer, used in SCAN_NEXTREQ to
360+
acknowledge the reception of the batch of rows from a fragment scan.
361+
If RNIL, this means that this particular fragment is done scanning.
362+
*/
350363
Uint32 tcPtrI;
364+
/*
365+
info encodes the number of rows and the length of the data sent in
366+
TRANSID_AI signals.
367+
*/
351368
Uint32 info;
352369
};
353370

@@ -415,10 +432,25 @@ class ScanTabRef {
415432

416433
};
417434

418-
/**
419-
*
420-
* SENDER: API
421-
* RECIVER: Dbtc
435+
/*
436+
SENDER: API
437+
RECIVER: Dbtc
438+
439+
This signal is sent by API to acknowledge the reception of batches of rows
440+
from one or more fragment scans, and to request the fetching of the next
441+
batches of rows.
442+
443+
Any locks held by the transaction on rows in the previously fetched batches
444+
are released (unless explicitly transfered to this or another transaction in
445+
a TCKEYREQ signal with TakeOverScanFlag set).
446+
447+
The fragment scan batches to acknowledge are identified by the tcPtrI words
448+
in the list of struct OpData received in ScanTabConf (scan fragment record
449+
pointer).
450+
451+
The list of scan fragment record pointers is sent as an array of words,
452+
inline in the signal if <= 21 words, else as the first section in a long
453+
signal.
422454
*/
423455
class ScanNextReq {
424456
/**
@@ -455,7 +487,12 @@ class ScanNextReq {
455487
UintR transId2; // DATA 3
456488

457489
// stopScan = 1, stop this scan
458-
490+
491+
/*
492+
After this data comes the list of scan fragment record pointers for the
493+
fragment scans to acknowledge, if they fit within the 25 words available
494+
in the signal (else they are sent in the first long signal section).
495+
*/
459496
};
460497

461498
#endif

‎storage/ndb/include/kernel/signaldata/TcKeyReq.hpp‎

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ class TcKeyReq {
9797
// Conditional part = can be present in signal.
9898
// These four words will be sent only if their indicator is set.
9999
// ----------------------------------------------------------------------
100-
UintR scanInfo; // DATA 8 Various flags for scans
100+
UintR scanInfo; // DATA 8 Various flags for scans, see below
101101
UintR distrGroupHashValue; // DATA 9
102102
UintR distributionKeySize; // DATA 10
103103
UintR storedProcId; // DATA 11
@@ -224,9 +224,18 @@ class TcKeyReq {
224224
/**
225225
* Scan Info
226226
*
227+
* Scan Info is used to identify the row and lock to take over from a scan.
228+
*
229+
* If "Scan take over indicator" is set, this operation will take over a lock
230+
* currently held on a row being scanned.
231+
* Scan locks not taken over in this way (by same or other transaction) are
232+
* released when fetching the next batch of rows (SCAN_NEXTREQ signal).
233+
* The value for "take over node" and "scan info" are obtained from the
234+
* KEYINFO20 signal sent to NDB API by LQH if requested in SCAN_TABREQ.
235+
*
227236
t = Scan take over indicator - 1 Bit
228-
n = Take over node - 12 Bits -> max 65535
229-
p = Scan Info - 18 Bits -> max 4095
237+
n = Take over node - 12 Bits -> max 4095
238+
p = Scan Info - 18 Bits -> max 0x3ffff
230239
231240
1111111111222222222233
232241
01234567890123456789012345678901
@@ -239,7 +248,7 @@ class TcKeyReq {
239248
#define TAKE_OVER_FRAG_MASK (4095)
240249

241250
#define SCAN_INFO_SHIFT (1)
242-
#define SCAN_INFO_MASK (262143)
251+
#define SCAN_INFO_MASK (0x3ffff)
243252

244253
/**
245254
* Attr Len

‎storage/ndb/include/ndbapi/Ndb.hpp‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@
6868
6969
If the operation is of type <var>Commit</var>, then the transaction is
7070
immediately committed. The transaction <em>must</em> be closed after it has been
71-
commited (event if commit fails), and no further addition or definition of
71+
commited (even if commit fails), and no further addition or definition of
7272
operations for this transaction is allowed.
7373
7474
@section secSync Synchronous Transactions
@@ -86,7 +86,7 @@
8686
- NdbTransaction::getNdbIndexOperation()
8787
- NdbTransaction::getNdbIndexScanOperation()
8888
along with the appropriate methods of the respective NdbOperation class
89-
(or one possiblt one or more of its subclasses).
89+
(or possibly one or more of its subclasses).
9090
Note that the transaction has still not yet been sent to the NDB kernel.
9191
-# Execute the transaction, using the NdbTransaction::execute() method.
9292
-# Close the transaction (call Ndb::closeTransaction()).
@@ -319,7 +319,8 @@
319319
either NdbScanOperation::updateCurrentTuple() or
320320
NdbScanOperation::deleteCurrentTuple()
321321
-# (If performing NdbScanOperation::updateCurrentTuple():)
322-
Setting new values for records simply by using @ref NdbOperation::setValue().
322+
Setting new values for records simply by using @ref NdbOperation::setValue()
323+
(on the new NdbOperation object retured from updateCurrentTuple()).
323324
NdbOperation::equal() should <em>not</em> be called in such cases, as the primary
324325
key is retrieved from the scan.
325326
@@ -346,7 +347,7 @@
346347
347348
@subsection secScanLocks Lock handling with scans
348349
349-
Performing scans on either a tables or an index has the potential
350+
Performing scans on either a table or an index has the potential to
350351
return a great many records; however, Ndb will lock only a predetermined
351352
number of rows per fragment at a time.
352353
How many rows will be locked per fragment is controlled by the

‎storage/ndb/include/ndbapi/NdbDictionary.hpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -995,7 +995,7 @@ class NdbDictionary {
995995
const char * getName() const;
996996

997997
/**
998-
* Get the name of the table being indexed
998+
* Get the name of the underlying table being indexed
999999
*/
10001000
const char * getTable() const;
10011001

@@ -1417,7 +1417,7 @@ class NdbDictionary {
14171417
struct RecordSpecification {
14181418
enum RecTypes {
14191419
AttrOffsetNotNULL= 1,
1420-
AttrOffsetNULL= 2,
1420+
AttrOffsetNULL= 2
14211421
};
14221422

14231423
enum RecTypes type;

‎storage/ndb/include/ndbapi/NdbIndexScanOperation.hpp‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -144,11 +144,22 @@ class NdbIndexScanOperation : public NdbScanOperation {
144144
/**
145145
* Marks end of a bound,
146146
* used when batching index reads (multiple ranges)
147+
*
148+
* With this call, it is possible to do multiple range scans (on the same
149+
* table/index) in a single readTuples() operation and roundtrip. Each range
150+
* bound (min and/or max) is defined with setBound(), and end_of_bound() is
151+
* called when all setBound() calls for one range bound have been done.
152+
*
153+
* The RANGE_NO argument is an application-selected value (in the range
154+
* 0-0xfff) that can be used to identify which range scan a particular row
155+
* belong to; the value can be obtained from get_range_no() when rows are
156+
* fetched with nextResult().
147157
*/
148158
int end_of_bound(Uint32 range_no);
149159

150160
/**
151-
* Return range no for current row
161+
* Return range no for current row, as defined in end_of_bound().
162+
* Only available if the SF_ReadRangeNo is passed to readTuples().
152163
*/
153164
int get_range_no();
154165

0 commit comments

Comments
 (0)