Date: prev next · Thread: first prev next last
2022 Archives by date, by thread · List index


At 14:44 05/06/2022 +1200, Steve Edmonds wrote:
I have just put a basic macro together so it will calculate income tax due from our local tax rates. I needed to round the tax to the nearest cent so taking the easy route I added Option VBASupport 1 to use the available VB round function. I then noticed that the VB round function with say round(12.125,2) rounds down to 12.12 and the LO inbuilt spread sheet function ROUND with ROUND(12.125,2) rounds up to 12.13. Is rounding in this situation arbitrary or is there some convention for consistency.

As is often the case, there are many conventions! See
https://en.wikipedia.org/wiki/Rounding#Rounding_to_the_nearest_integer .

The LibreOffice function is likely to use "round half away from zero" - the version we all learned in elementary school. But a quick web search reveals that the Visual Basic function does indeed use "round half to even" ("banker's rounding") - which explains the difference. You should find that both functions agree that round(12.135,2) is 12.14.

Note also two other matters:

o What is displayed in a spreadsheet as 12.125 may represent a slightly different underlying value - either more or less than this - with the display limited to three fractional places by cell formatting. The spreadsheet ROUND() function may act on the displayed value and give the expected answer, or it may act on the true value stored in the cell - which may round differently. Which of these happens is controlled by the option at Tools | Options | LibreOffice Calc | Calculate | General Calculations | Precision as shown.

o Some values which terminate as decimal fractions may not be capable of exact representation in the computer's binary form. This is likely to make a difference only when values are expressed to a significance near that of the underlying number storage. (And it happens anyway that your 12.125 *will* be capable of exact representation in binary.)

I trust this helps.

Brian Barker



--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/users/
Privacy Policy: https://www.documentfoundation.org/privacy

Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.