Lattice reduction in 3-dimensions with real basis vectors

Semaev's paper A 3-Dimensional Lattice Reduction Algorithm, linked in the question, actually does work when the basis vectors are in $\mathbb{R}^3$; the algorithm does not require integer vectors. The same holds for LLL: you can run the algorithm just fine for vectors in $\mathbb{R}^n$. In fact there is a lot of work done in this direction, adapting LLL to work better with lattices in $\mathbb{R}^n$: see for instance these articles on Floating point LLL by Stehlé and by Nguyen-Stehlé, and the Github repository for the implementation fplll.

All that said, there are a few important reasons why it's common to restrict to lattices in $\mathbb{Q}^n$ or $\mathbb{Z}^n$:

• Runtime analysis. Bounding the runtime often works by showing that some positive integer quantity decreases at each step of the algorithm, and therefore it must run out in a predictable number of steps. It's harder to make this sort of descent argument work if you replace this quantity with a positive real number: this measured quantity could decrease forever without ever running out. In practice, this means the algorithm may run very quickly for integer basis vectors, but could take significantly longer for real basis vectors of a similar size. (Semaev restricts to integer vectors in his runtime analysis for this exact reason.)
• Error propagation. Computing with arbitrary real numbers can't be done exactly. Whether you use floating point arithmetic, interval arithmetic, or some other implementation of real number arithmetic, as you run the algorithm, the quantities you compute will be less and less precise. (See the Wikipedia article on Floating-point error mitigation for a brief intro to how tricky it can be to manage these errors.) With integers/rationals, on the other hand, the operations are all exact.
• Every lattice is "essentially" an integer lattice. If you have some lattice in $\mathbb{R}^n$, you can't store all infinitely many decimal digits of the vector components anyways, so in practice you might just store, say, the first $20$ digits of each component of each vector. But this means your vectors are actually in $\mathbb{Q}^n$. And if you scale the lattice up by $10^{20}$, the scaled vectors are in $\mathbb{Z}^n$. So you may as well just use the integer algorithm on your new integer lattice, and then scale down the result when you're done. This is what's usually done in practice: instead of using real number arithmetic, where you have to work a bit harder to deal with the issues mentioned above around runtime and error accumulation, it's often much more straightforward to just approximate your lattice by a rational one and apply the more predictable error-free algorithms (though some care needs to be taken with how the rounding is done, especially as the dimension $n$ grows). See the Application section of the LLL Wikipedia page for example, where they define basis vectors using the golden ratio rounded to the nearest millionth.