I am working on a viewshed* algorithm that does some floating point arithmetic. The algorithm sacrifices accuracy for speed and so only builds an approximate viewshed. The algorithm iteratively ...

An unfortunate reality of trying to represent continuous real numbers in a fixed space (e.g. with a limited number of bits) is that this comes with an inevitable loss of both precision and accuracy.

Recently I've become involved in multiple projects where inherent limitations in floating point numbers are leading to problems (for example, carry out a few hundred matrix multiplications and the ...

Why floating point is important for developing machine-learning models. What floating-point formats are used with machine learning? Over the last two decades, compute-intensive artificial-intelligence ...

Hardware for integer or fixed-point arithmetic is relatively simple to design, at least at the register-transfer level. If the range of values and precision that can be represented with these formats ...

In a recent survey conducted by AccelChip Inc. (recently acquired by Xilinx), 53% of the respondents identified floating- to fixed-point conversion as the most difficult aspect of implementing an ...

In 1985, the Institute of Electrical and Electronics Engineers (IEEE) established IEEE 754, a standard for floating point formats and arithmetic that would become the model for practically all FP ...

Embedded C and C++ programmers are familiar with signed and unsigned integers and floating-point values of various sizes, but a number of numerical formats can be used in embedded applications. Here ...