kaldi/src/tensor/pattern.h at kaldi10 · feelingstack/kaldi

Repository navigation

History

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

469

470

471

472

473

474

475

476

477

478

479

480

481

482

483

484

485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

620

621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

639

640

641

642

643

644

645

646

647

648

649

650

651

652

653

654

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

// tensor/pattern.h

// See ../../COPYING for clarification regarding multiple authors

// Licensed under the Apache License, Version 2.0 (the "License");

// you may not use this file except in compliance with the License.

// You may obtain a copy of the License at

// http://www.apache.org/licenses/LICENSE-2.0

// THIS CODE IS PROVIDED *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY

// KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED

// WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A PARTICULAR PURPOSE,

// MERCHANTABLITY OR NON-INFRINGEMENT.

// See the Apache 2 License for the specific language governing permissions and

// limitations under the License.

#ifndef KALDI_TENSOR_TENSOR_PATTERN_H_

#define KALDI_TENSOR_TENSOR_PATTERN_H_ 1

#include "tensor/tensor-common.h"

#include <limits>

namespace kaldi {

namespace tensor {

PATTERN GLOSSARY (note: see also TENSOR GLOSSARY in tensor.h)

Adjacent: Two Patterns are said to be adjacent if their memory-index-sets

are disjoint and a Pattern exists whose memory-index-set is

their union.

[TODO: come up with algorithm for testing adjacency and

merging the adjacent Patterns.]

Axis: An axis is one of the (dim, stride) pairs that form part

of a Pattern. We will sometimes use the word "axis"

to refer to the integer index of the axis, as in, for example,

in a Tensor with dims=[5 6 7], axis 0 has dim=5 and

axis 2 has dim=7; but this should more precisely

be called axis-index or raxis-index (see their own

glossary entries; they respectively use the public

numbering, or reversed private numbering). To describe

the number of axes of a Tensor, we use the term "num-axes" /

"number of axes".

Axis-index: An axis-index of a Pattern or Tensor (sometimes just "axis" for short,

especially in code) is an index that identifies an axis in the

public numbering (see "Public numbering"). A valid

axis-index for a Pattern with `num_axes` axes is in the

range [0, num_axes - 1].

For an axis-index i, the corresponding raxis-index (c.f. "Raxis-index:"

or "Private numbering:") would be num_axes - 1 - i.

See also "Eaxis-index" for where we allow negative axis-indexes

as offsets from the end.

axis-dominance property: search below for [Valid Pattern], point (v), for the main

definition.

[axis-dominance property of an axis-index]:

There is another sense in which we use the term

'axis-dominance property': for a Pattern whose axes are sorted

from least to greatest abs(stride) [in the private numbering],

we say that "the axis-dominance property holds for axis-index r

of that Pattern" if:

dim(r) * abs(stride(r)) <= abs(stride(r+1)).

[axis-dominance lemma]

The axis-dominance lemma, of which we won't provide a proof

of here as it's pretty obvious, is something you would need

when showing that axis-dominance implies uniqueness. It

states that, for a pattern which is valid-1, for any

any 0 <= r < num_axes,

(\sum_{q < r} (dim(q) - 1) * stride(q)) < stride(r).

Broadcasting: A convention whereby for an operation on Tensors that would

normally be required to have the same dimension, it's

acceptable for, on some axis, one Tensor to have `dim = n`

with `n != 1` and the other to have `dim = 1`. I.e., two dims can be

different as long as one of them is 1. Most operations will

take place as if the Tensor with `dim = 1` had been extended

to `dim = n` by making identical copies. However, if it is

the output Tensor that has `dim = 1`, there would be summation

or possibly some other appropriate reduction instead of making

copies. This is different from other toolkits (the fact that

we extend the concept of broadcasting to encompass summation).

See also: Broadcastable (which has a more precise definition);

PyTorch-style broadcasting, extended indexing.

Broadcastable: See documentation for function Broadcastable() in pattern-utils.h.

Explaining it in terms of the public numbering: two

Patterns are broadcastable if their dims (padded as

necessary on the left by 1's to make them the same size)

are, for each axis, either the same or one of them is 1.

So for example, comparing ([ 3 4 ], [4]), we first pad on

the left to get ([3 4], [1 4]); then we say they are

broadcastable because 4 == 4 and in the remaining axis, one

of the dimensions is 1.

Canonical form: A Pattern is in canonical form if all pairs of axes that

could be combined (without affecting its memory-index-set)

have been combined; where there are no trivial axes; all

strides are positive; and the axes are sorted in an order

of stride that's increasing in the private numbering /

increasing in the public numbering.

See CanonicalizePattern().

Compact: A Pattern is compact if its memory-index-set forms a contiguous

range of integers (no gaps). (We don't call this "contiguous"

because PyTorch uses the same word with a different meaning).

Default strides: The default strides for a pattern with provided dimensions are:

of course, zero for any axis with dim=1; and otherwise (describing

it in the public numbering of axes), each axis's stride is

the product of the later-numbered axes' dims. It corresponds

to the strides of a "C" array.

This is the policy that we will use when constructing new

Tensors if only the dims are provided, which is why we call these

the default strides.

A Pattern having default strides is equivalent to its having

normalized strides and also being compact.

See also: raxis-index

PyTorch-style broadcasting: We use this name to refer to the fact that in

PyTorch, if an operation is done on two Tensors with

dims=[5 6] and dims=[6], the second one would be interpreted

as having dims=[1 6]. That is: we pad with 1's on the left

(See "Padding"). Note: whenever we refer to broadcasting

we include this feature; this glossary entry exists just

to explain it, not to imply that we have two different

versions of broadcasting.

Raxis-index: We use the term "raxis-index", often just "raxis" for short,

to mean the index of an axis in the reversed, private numbering.

This would usually be in the range [0, num_axes - 1] for

a Pattern with `num_axes` axes, but for broadcasting purposes,

if we are doing an operation between Tensors of different

numbers of axes we may often use larger raxis values for the Tensor

of smaller num_axes (see PyTorch-style broadcasting).

Set-equivalent: Two Patterns are set-equivalent if their memory-index-sets

are identical.

Two Pattern-tuples are set-equivalent if their

memory-index-tuple-sets are identical.

Shape of a Pattern: The vector of the dimensions of a Pattern: e.g. [] for

a Pattern with num_axes = 1 or [2 3] for a Pattern with

num-axes = 2. Note: whenever we display dims vectors in

square brackets or use "shape" without qualification, it

implies we are displaying them in the public numbering.

The "private shape" of a Pattern are be the same in the

private numbering, which is in the reverse order to

the public numbering and which we'd display in curly

braces, like {3, 2}.

Shape of a Pattern-tuple: The shape of a Pattern-tuple is

formed by taking the shapes of each Pattern in the tuple,

extending them on the left with 1's as necessary to make

them the same size, then taking the largest dim on each axis

(i.e. the one that is not equal to 1, if they are

different). For example, for a Pattern-tuple of Patterns

whose shapes were ([4 1 5], [6 1], [5]), the shape of the

tuple would be [4 6 5]. (Note: the Patterns in a

Pattern-tuple must be broadcastable, so if the dims are

different, one of them must be 1.)

Trivial axis: An axis of a Pattern for which dim=1. Such axes will have

stride=0 if the Pattern is valid.

Memory-index-set of a Pattern:

The memory-index-set M(p) of a Pattern p is

the set of all memory-indexes obtained by indexing

the pattern with all index-tuples in the index-tuple-set

I(p) of the Pattern. By extending the notion of indexing

a Pattern (c.f. "Indexing a Pattern") to take set

arguments, this could be written as M(p) = p[I(p)]. Note:

by the uniqueness property, we always have |M(p)| = |I(p)|

for a valid Pattern, i.e. the sizes of the sets are the

same.

Memory-index-tuple-set of a Pattern-tuple:

The set of all memory-index-tuples M(P, Q) obtained by indexing

the Patterns in the tuple (P, Q) with all members of the

index-tuple-set of the Pattern-tuple. See "memory-index-tuple"

and "index-tuple-set of a Pattern-tuple" for more information.

View the notation M(P, Q) as shorthand for M((P, Q)).

Normalized strides: We say that a Pattern has normalized strides if the

strides are all nonnegative and the nonzero strides

are in strictly increasing order in the private numbering

(hence strictly decreasing in the public numbering).

See also: Default strides (which is a stronger property).

Normalized+ strides: Normalized+ strides are strides that are normalized

but also positive. I.e. the strides are positive, and

increasing in the private numbering / decreasing in

the public.

Linear property:

This is a slightly technical property used in certain

proofs involving patterns.

Consider patterns P and Q with the property that the

memory-index-set of P is a subset of the memory-index-set of

Q. If i is an index-tuple, let P(i) be the map from

i to a memory-index, and let

\f$ Q^{-1}(m) \f$

be the function that maps a memory-index m in the memory-index-set

of Q to the index-tuple i in the index-tuple-set of Q such

that Q(i) = m. Then we say that P is linear in Q if

for all index-tuples i and j such that i, j and i + j are

in the index-tuple-set of P,

\f$ Q^{-1}(P(i)) + Q^{-1}(P(j)) = Q^{-1}(P(i+j)) \f$.

[Transitivity]

It is easy to show that the linear property is transitive;

that is if P is linear in Q and Q is linear in R, then

P is linear in R.

Reduced pattern:

A pattern is in reduced form if there is no

set-equivalent pattern which has fewer axes.

What this means more concretely is that the pattern has no

trivial axes and has no pairs of axes which could be

combined. For example, a matrix where successive rows

"touch" (i.e. they are not separated by a stride) can

always be reduced.

Reduced pattern-tuple:

A pattern-tuple is in reduced form if there is no

set-equivalent pattern-tuple which has fewer axes

(defining the num_axes of a pattern-tuple as the

greatest of the num_axes of the patterns in the tuple).

What this means more concretely/intuitively is that there

are no axes which are trivial for all patterns and can be

removed; and there are no pairs of axes which can be

combined for all patterns in the tuple.

Regularity property: This is a property of Patterns that is relevant when

reducing Patterns to a common set of strides. It can

be thought of as a relaxed version of the axis-dominance

property.

We formulate the regularity property to only apply for

Patterns which are valid-2 and which have positive strides

in increasing order. The stipulation on having postive,

sorted strides is just for convenience, since we happen to

need this property only in that case and it's easier to

formulate in that case.

A Pattern is regular if, in addition to satisfying the

properties mentioned above, for each axis-index

0 <= i < num_axes - 1,

there is an integer k with i < k <= num_axes, such that:

(i) For all j with i < j < k, stride(i) divides stride(j)

exactly and dim(j) = 1.

(ii) Either k == num_axes, or dim(i) * stride(i) <= stride(k),

The reader may notice that if we were to restrict k to

equal i + 1, then this would be equivalent to the

axis-dominance property (property (v)) plus the

requirement that the strides be positive and sorted (which

we only added for convenience).

Storage region: A Tensor, in addition to a Pattern, has a storage region

that can be though of as a pointer (say, to float) which

we index with a memory-index: say, p[m], if s is the

pointer and m is the memory-index. See storage.h.

See also "Dereferencing a memory-index".

Stride: A stride is the distance, in elements, between successive

elements of a Tensor along a particular dimension.

For example, a Tensor with one axis having dim=3 and

stride=2 would have its elements laid out in memory

as: `[ element0 xxx element1 xxx element2 ]`,

where `xxx` means an element that is not part of the

Tensor. Axes with dimension=1 always have stride=0

in this toolkit. Tensors with negative strides may be created,

although they will be copied to temporaries with

positive stride in linear algebra operations where

necessary (since most BLAS implementations do not support

negative stride).

Uniqueness property: A property of a Pattern that there does not exist

two index-tuples i1 and i2 which are different in non-trivial

axes of the pattern (i.e. i1[r] != i2[r] for some r that is

a non-trivial raxis of the pattern), which when

used to index the Pattern, generate the same memory-index.

The axis-dominance property is sufficient, but not

necessary, to ensure the uniqueness property. (The

uniqueness property is probably not easy to test for

efficiently in the general case where axis dominance does

not hold).

Valid Pattern:

A valid Pattern must be as follows. Think of this as the mathematical definition;

see the declaration of struct Pattern for additional details about how

it is stored.

(i) The num_axes must satisfy 0 <= num_axes <= KALDI_TENSOR_MAX_DIM

(ii) The dims must all be >0a

(iii) We require dim[i] == 1 and strides[i] == 0 for

num_axes < i < KALDI_TENSOR_MAX_DIM

(iv) We require that no memory index reachable through the pattern

be negative, which can be expressed as:

offset + \sum_{i=0}^{num_axes - 1} min(0, strides[i]*(dims[i]-1)) >= 0

(v) The strides must be nonzero for axes i with dim[i] != 1.

(vi) the axis-dominance property. This property is sufficient, but not

necessary, to ensure the uniqueness property. It requires that

when the axes are sorted from least to greatest value of abs(stride),

for each axis-index 0 <= r < num_axes - 1 (using the private numbering

of axis-indexes),

dim(r) * abs(stride(r)) <= abs(stride(r+1)).

(Note: this property doesn't require that the axes be sorted that

way; if you need that, search for "Canonical form").

(vii) the strides must be zero for axes with dim=1.

Valid-1 Pattern:

A Pattern is valid-1 (read as: valid minus one) if it

satisfies properties (i) through (vi) of a valid Pattern

(i.e. it may have nonzero strides for axes with dim=1, but

must otherwise be valid). A valid pattern is also valid-1.

Valid-2 Pattern:

A Pattern is valid-2 (read as valid minus two) if it

satisfies properties (i) through (v) of a valid Pattern

and also satisfies the uniqueness property. That is, it must

be a valid Pattern, except:

- it may have nonzero strides for axes with dim=1, since

we don't require property (vi)

- it does not have to satisfy the axis-dominance property

(property (vi)).

However, it must still satisfy the uniqueness property

(see its glossary entry); we don't normally explicitly

require the uniqueness property because it is implied by

the axis-dominance property.

A pattern that is valid or valid-1 is also valid-2.

This struct stores the dimension and strides of a Tensor.

Below we describe the the properties that a Pattern is required to have.

Most of them are described in the glossary in the entry for "Valid Pattern",

but there are a couple more that have to do with the specifics of how we

store things in this struct.

These properties are stricter than some other frameworks, such as PyTorch,

which allow the users to manually add dimensions with stride 0 and dim > 1 so

that a lower-dimensional quantity can masquerade as one with a higher

dimension. (This framework allows the same kinds of operations, they are just

not done by the same mechanism). We

also don't allow zero dims (i.e. a Tensor that is initialized must not have

num_elemnts==0). If you want an empty Tensor, just use a null pointer. In

addition, we require that the stride equal zero for any axis that has dim = 1.

There is also the "axis-dominance" property (see its glossary entry for more info).

Our requirements of a Pattern are:

0 <= num_axes <= KALDI_TENSOR_MAX_DIM.

for 0 <= i < num_axes:

dims[i] > 0

if dims[i] == 1, then strides[i] = 0.

if dims[i] != 1, then strides[i] != 0

for num_axes <= i < KALDI_TENSOR_MAX_DIM:

dims[i] == 1

strides[i] == 0

offset >= 0

The axis-dominance property (see property (v) in "Valid Pattern" above)

Note: in the public interface of class Tensor, if you ask for Dim(i) it will

return pattern.dims[pattern.num_axes - i], i.e. the interface uses the public

numbering, while the axes are physically stored using the reversed "private

numbering". This reversal makes it much easier to implement

PyTorch-style broadcasting where in an operation on Tensors of dims,

say, (3,4) and (4), the (4) is interpreted as (1,4).

struct Pattern {

int32 num_axes;

int32 dims[KALDI_TENSOR_MAX_DIM]; // the dims in reversed order, indexed

// by 'raxis' (reversed axis)

int32 strides[KALDI_TENSOR_MAX_DIM]; // the strides in reversed order,

// indexed by 'raxis' (reversed axis)

int64 offset; // Offset of the element with all-zero indexes

// from the start of the originally allocated memory

// region

int32 code; // pattern code; from user-level code it should be accessed via

// GetCode(), which ensures it is set. See documentation for

// ComputePatternCode() in pattern-utils.h for details of what

// this represents. If this is negative then it means it has not

// been computed. In a valid Pattern the code will always be

// either negative or up-to-date; GetCode(), which assumes the

// Pattern was valid, computes the code if it was negative.

int32 properties; // More occasionally-needed properties. This is similar to

// OpenFst's notion of properties, where we compute them

// only on demand. In a valid Pattern the properties

// will always be accurate, but see "Accurate properties"

// in glossary above for definition (it can be zero).

// Returns true if the Pattern is valid. This includes all the

// mathematical conditions on a valid Pattern (search above for "Valid

// Pattern"), plus extra conditions related to struct Pattern,

// namely: dims and strides with index >= num_axes should be

// 1 and 0 respectively; and the code should either be -1 or or

// be the same as ComputePatternCode() returns on this pattern.

// See also IsCanonical() in pattern-utils.h.

bool IsValid();

// Returns the pattern's code (the correct code, not -1). Requires

// the Pattern to be valid (and in a valid Pattern, the code must be -1

// or the correct code, so we may assume a code >= 0 is the correct one.

int32 GetCode();

// This comparator induces a total ordering on valid Patterns. It is a

// lexical comparison on the offset, num_axes, dims and strides. (The code

// does not need to be compared because, if not -1, it is a function of the

// dims and strides).

bool operator < (const Pattern &other) const;

// Equality operator on Pattern. Compares the num_axes, offset, and dims and

// strides indexed [0... num_axes-1]. (In patterns that satisfy IsValid(),

// the remaining dims and strides would be 1 and 0 respectively, so checking

// them is pointless).

bool operator == (const Pattern &other) const;

inline bool operator != (const Pattern &other) const {

return !(*this == other);

}

// Assignment operator (copies all members).

bool operator = (const Pattern &other) const;

};

/// Returns a string representing a Pattern, of the form:

/// "offset=a dims=[b c d] strides=[e f g]"; this is for debugging

/// purposes.

std::string PatternAsString(const Pattern &pattern);

/// Returns a string representing the dims of a Pattern, something like

/// "[10 20 100]"

std::string DimsAsString(const Pattern &pattern);

/// Returns a string representing the strides of a Pattern, something like

/// "[1 10 200]"

std::string StridesAsString(const Pattern &pattern);

// We may later get rid of this struct and just have functions to get

// these properties.

struct PatternProperties {

// Below are cached properties that are derived from the underlying data in

// struct Pattern.

// The number of elements in the Tensor, which equals the product

// of dims[0] .. dims[num_axes - 1]. Will always be >0.

int64 num_elements;

// Binary code describing the pattern, see GetPatternCode() in

// pattern-utils.h.

int32 code;

// is_contiguous means that the data form a contiguous block in memory; it is

// not the same as PyTorch's is_contiguous which is a stronger condition,

// implying also that the strides are as for a `C-style` array.

// TODO: see if this is even needed; it may not be.

bool is_contiguous;

// has_c_strides means that the strides of all axes i with dim[i] != 1,

// equal the product of all later-numbered dims, i.e.

// \f$ strides[i] = \prod_{j>i} dim[j] \f$, or `strides[i] = 0` if

// dim[i] == 1 (since we use the convention that axes with dim=1 always

// have stride=0.

// has_c_strides is the equivalent of PyTorch's is_contiguous.

// this->has_c_strides implies this->is_contiguous.

// TODO: see if this is even needed; it may not be.

bool has_c_strides;

// Sets the members of *this to be the properties of pattern 'pattern'.

// Ignores the previously existing values of *this.

void UpdateProperties(const Pattern &pattern);

};

/**

Returns a hash value for hashing Pattern. Depends on num_axes,

offset, and dims and strides indexed [0... num_axes-1]. pattern does

not have to be valid.

size_t GetHash(const Pattern &pattern);

// C++ hashing object for Pattern

struct PatternHasher {

size_t operator (const Pattern &pattern) { return GetHash(pattern); }

};

// C++ hashing object for Pattern*; requires the pointer

// be non-NULL and to point to a Pattern.

struct PatternPtrHasher {

size_t operator (Pattern *pattern) { return GetHash(*pattern); }

};

struct PatternPtrEqual {

size_t operator (Pattern *pattern1,

Pattern *pattern2) {

*pattern1 == *pattern2;

}

};

} // namespace tensor

} // namespace kaldi

#endif // KALDI_TENSOR_TENSOR_PATTERN_H_

FilesExpand file tree

pattern.h

Latest commit

History

pattern.h

File metadata and controls