Date: prev next · Thread: first prev next last
2022 Archives by date, by thread · List index



On 6/5/2022 6:54 AM, Michael H wrote:
Are you sure it isn't somewhere else?  I.e. the input to the actual round
step from the vb and the LO function?  LO automatically shortens numbers
and rounds them as it does so... so a that shows 12.15 may actually contain
12.14999999 or 12.1500001

This whole discussion reminds me of a problem I had some 30+ years ago when I was the treasurer for my children's preschool. Every week, when I prepared the paychecks, one of them kept coming out one penny short. At the time, I used an old DOS spreadsheet to calculate the paycheck, and I had created a long complex formula to compute the paychecks.

At one point in the computation, the computer had to divide 28 by 7. Now every third grader knows that 28 divided by 7 equals 4, but for some reason, my spreadsheet formula gave a result of 3.9999999999, and that answer, when combined with other steps in the computation, resulted in a paycheck one cent short. A computer friend of mine said that my program was being more precise, but in my simplistic brain, I couldn't see how a wrong answer was more precise than a correct answer.

Oddly enough, when I switched to a Basic program on my antique Tandy Model 100 laptop, I got the correct answer and all was right with the paychecks.

Virgil



--
To unsubscribe e-mail to: users+unsubscribe@global.libreoffice.org
Problems? https://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: https://wiki.documentfoundation.org/Netiquette
List archive: https://listarchives.libreoffice.org/global/users/
Privacy Policy: https://www.documentfoundation.org/privacy

Context


Privacy Policy | Impressum (Legal Info) | Copyright information: Unless otherwise specified, all text and images on this website are licensed under the Creative Commons Attribution-Share Alike 3.0 License. This does not include the source code of LibreOffice, which is licensed under the Mozilla Public License (MPLv2). "LibreOffice" and "The Document Foundation" are registered trademarks of their corresponding registered owners or are in actual use as trademarks in one or more countries. Their respective logos and icons are also subject to international copyright laws. Use thereof is explained in our trademark policy.