Precision Definition and 126 Threads

Is it possible to punch 1mm holes in a sheet of SS 316 of thickness 3mm. I have a CNC punching machine but not sure if this would be possible.

Hello! (Note : I put the prefix as comp sci, since at my uni this is a computer science class but I am in EE so it could be put under both prefixes. If anyone feels its more appropriate as Engineering feel free to change) So here is my attempt at the solution The formula that we are given is...

First off, I will set out what I think I know so that any misconceptions of mine can be put right. Definitions: Precision: a quality denoting the closeness of agreement between (consistency, low variability of) measured values obtained by repeated measurements Accuracy: A quality denoting the...

I am developing a Taekwondo body protector scoring system using a textile piezoelectric sensor. The sensor is designed to detect mechanical force and convert it into an electrical charge. However, it is sensitive to temperature variations, leading to potential inaccuracies and instability in the...

I am new to statistics and recently learned about ISO guidelines for Accuracy & Precision and Uncertainty & Error. But there are some graphs of probability distribution I found on internet which I am not able to grasp. image Source Q. In this graph(above) if...

Hello! I need a voltage divider where the input is between -10 and 10 V and the output is between -4 and 4 V. I would need something with a bandwidth of 1 (maybe even 10) MHz. Can someone recommend me a reliable product (the price plays a role, too, so being cheaper would be also useful)? Thank you!

Hi, I'm looking to improve the precision of a system over a several hour cycle of operation of +50°C operation. The mass is ~11g and is suspended below the balance via a factory provided point. The "sample" is enclosed within a large borosilicate flask 150mm dia x 450 mm height that is trace...

I know that this expression evaluates to 1 when a is equal to 0. Also for when a is equal to 1/n when n is a positive number, but I'm confused about how to go about this in double precision?

Hello, When we measure something, using a measuring instrument, our measurement will be a number with a finite number of significant figures. The rightmost digit is the uncertain digit affected by uncertainty. A measurement should always be reported as the best estimate +-error range where the...

Hello! In many precision experiment, especially those overlapping with quantum computing techniques, such as trapping a few ions in a Paul trap, they use cryogenic systems (around 1K). I am not totally sure I fully understand the advantage of that, compared to room temperature. For example, a...

Hi, I have $150. This value is multiplied by 2 or more factors. Each factor is less than or equal to 1 and greater than or equal to 0. Each factor has a maximum of 9 decimal places. The sum of the factors equals 1. For example, the following set of factors meets these 3 conditions: 0.500033333...

What is the most precise way to log temperature in the laboratory? I am concerned with resolution and precision here, not accuracy. RTDs don't seem to cut it. Almost all that I have looked at and tried only have a resolution of around 0.1 deg C. I was hoping for 10X better than that- Thanks...

NOTE: this is a programming exercise (Python). I started adding to ##x_{true}## an error related to a (for example) 10% relative error, obtaining ##x_{measurement}##. Then i computed ##y_{measurement}##. To find the precision, i calculated ##(y_{true}-y_{measurement})/y_{measurement}##. If it is...

Hello. I need help to understand why my code is not giving me double numerical precision. In this piece of code, I create a function that is analytically normalized, but when I calculate the numerical normalization factor, it seems to be with single precision. Here it is: program dyna...

If you read any introductory physics or chemistry course, in the very first chapter, you'll come across measurement, uncertainty, etc. There, you learn some rules about rounding off numbers, etc. One of the rules discussed there is that if you add/subtract two numbers with different decimal...

Homework Statement Two different scientists measured the speed of light four times. Scientist A 3.80*108m/s 3.32*108m/s 3.10*108m/s 3.05*108m/s Average 3.32*108m/s Scientist B 2.90*108m/s 3.10*108m/s 3.24*108m/s 2.95*108m/s Average...

Hi, Mostly i work with Octave / Matlab but i am trying to get into python also. Lately i have a couple problems where my numbers can't be represented in binary64 or float64 default format because they exceed the max of 1.8x10^308. Is there a common way people deal with this (without toolkits)...

The following program is printing wrong value of RWTSED. How can I print correct value?? program inpdat c IMPLICIT NONE REAL RHOMN,RWTSED,VOLSED VOLSED = 17424.0 RHOMN = 2.42 !0000076293945 RWTSED= VOLSED*RHOMN*1.0E6 2000 FORMAT(/,3F40.5)...

Hi, I was watching a really interesting old video from Richard Feynman who was talking about how precise/accurate the predictions of quantum theory agree with observation. The number of decimal places it agrees he said is astounding. He likened it to measuring the circumference of the Earth to...

Can someone explain the difference between trueness and bias, as used in measurement?

The website of CIAAW list the 2012 atomic mass in Daltons (Atomic Mass Units) and mention the atomic mass of stable isotopes followed by an additional number between braces. For example Helium-4: 4.002 603 2541(4) My first question is what does the number between braces mean? I assume it is some...

Ever when I see one of these beautiful brass scientific instruments, like microscopes, telescopes, goniometers etc. from the 19th century, I wonder how they have been actually made and whether I could learn how to do this. I think that most of gears used in the construction of these pieces were...

Homework Statement We performed an experiment to see which laboratory glassware was more accurate and which was more precise. Question: (a): Which glassware was more precise and why? (b) Which glassware was more accurate and why? Use the calculations to support your statement. I won't post the...

It is claimed that QED could predict certain quantities, like the lamb shift and magnetic moment of electron with astonishing precision of 12 digits. My question is the following: does QED predict only a few things to this precision, or does it predict, this precisely, the entire arbitrary...

Homework Statement [/B] Over 40 years ago, the Apollo astronauts placed reflectors on the surface of the Moon. These are still used by a number of observatories on Earth to monitor the distance to the Moon by reflecting pulses of laser light from them and detecting the reflected signal...

Supposing we have 2 circles in a 2 d space that can bounce off each other like balls. These circles are made of an infinite number of points. We put the centre of each circle on a line and send them towards each other along the line. Are they going to bounce off keeping their trajectory on the...

fresh_42 submitted a new PF Insights post Representations and Why Precision is Important Continue reading the Original PF Insights Post.

Hello I am currently using double type for numbers with the format 6f8 (6 digits before the point, 8 digits after the point). Double structure is enough for this format, or should I use int64_t instead? Thank you in advance :) Greetings. PS: I tried with this example code, but the result...

Homework Statement Some sources say the implied precision of a measurement is half of the least significant figure, while others say the implied precision is "1" in the place of the least significant figure. For example, a length is measured as 3.28 m. According to some sources, the implied...

Any suggestion on a good (but as usual not that expensive) scale that can measure at least to centigram precision? Searching online (e.g. Amazon) I did find some options but the reviews are usually mixed regarding precision (users will say that scales with milligram precision specs are not that...

Hello, I have an exercise here that I need help with. The precision in measurements of radial velocities by the Doppler effect is currently 1 m/s. Can a Jupiter like planet orbiting a star similar to the Sun at a distance from the mother star equal to the Sun-Jupiter distance be detected? (Use...

Is precision the smallest division in measurement instruments? Is uncertainty one-tenth or a half of the smallest division in measurement instruments? I'm confused some say one-tenth, some say a half... Thanks in advance

Hello friends, today i have this question about precision. Actually I'm trying to do a simple use of the pow function for a small number x=0.005 squared. By entrying pow ( x, 2) my console does not output the proper number 0.000025, but some memory error. I've read about it and i may be able to...

I learned Fortran from ancient people who disregard modern standards. As such, my code consists mainly of that older syntax. I am wondering how well this meshes with current Fortran environments, specifically regarding double precision calculations. This comes up mainly because I have been...

Feynman's approach to precision QED calculations, see his book "Quantum Electrodynamics", was shown(?) to be equivalent to the method of quantum fields. Both methods get the right the get the right answers to some problems. What quantum problems does Feynman's method (there is a name for that I...

It seems that the value has only 1 1/2 significant digits. Why are values for the measurement of this radius not more precise? What prevents a greater precision? I have recently been trying to learn about the sun's orbit in the Milky Way (MW) in order to calculate estimates re DM for another...

Homework Statement You are asked to find the weight of a paper clip as accurately as possible and given ten paper clips and a scale that has a precision of 0.1 g. (A paper clip weighs about 1 g.) Which of the following proceedures would give you the most precise measurement the weight of a...

Hi all, Here's my first post with a question I've been trying to solve for a while. I work with electronic components of very high quality and we try to make all our connections using thin copper ribbons of various thickness. I have started cleaning my copper connections with a solution of...

Homework Statement I've got a doppler shift question about calculating the frequency heard by a receiver moving 5.0m/s towards a sound source that is emitting 3000Hz sound. I assumed speed of sound is 343 m/s for an air temperature of 20C. Homework Equations Freceiver = Fsender((V +...

I'm designing a fixture for quality control and need a few precision holes. Their diameters range from 1/8" to 5/16" and need to be ±0.0002" with a depth of 1/2". I've been looking into EDM hole making and honing, but neither seem to be great for that range of diameters. Anyone know of any other...

So I have a header file containing things like #ifndef _params_h #define _params_h #define REAL double // precision of data #define ALIGN 64 // byte size of alignment #define DBG 1 // 1 prints out results #define READIN 1 // 1 reads in white paper data, 0 randomizes data #define...

Could someone please show me some examples how IEEE single precision format can used to store approximately 7 decimal digits of a number when the number is written in decimal format? How this is related to the statement ##2^{-23}=1.19\times10^{-7}##?

If you are given a pure sinusoidal wave and a perfect DC is required from it. then what is the best way to get it? should we go for full bridge rectifiers? or any better way is there?

What limits the precision with which you can describe the uncertainty of a measurement? I will describe two examples that feel qualitatively different, but I am not sure if they are quantitatively different in how you would deal with the uncertainty. 1) Measuring the length of a pencil with a...

Hey all, So I've been writing a hard science fiction novel and need real answers from someone who understands the fields better than I ( I have no formal education outside basic high school classes in physics) The problem I have is I'm not a fan of faster than light travel but I need a way for...

Hello I have been trying to find out how the data for the atomic mass of different isotopes is processed. I am vaguely familiar with mass spectrometry and other techniques used for the measurements and I know that the experimental results are coordinated by the Atomic Mass Data Centre. I am...

Homework Statement so let's say that we have a solution S=0.4±0.01/250ml i want to do a dilution so i pick 5ml and pour it into 100ml beaker how should i calculate precision of the solution (mathematically or with some other equations)? Homework Equations so basically we have F=5*S/100 The...

Hi all, I am currently studying OpAmps and one of the topics that came up was the applications of OpAmp circuits. In particular, I was learning about precision rectifiers (using non-inverting version of it, thus the diode is pointing away from the "out" terminal of the OpAmp). The thing I...

Homework Statement I'm doing a quality of measurements coursework at the moment where I have to find the efficiency of the different volumes of water. To do this, I needed to use this formula: After measuring the efficiency of different volumes of water, I'm asked to assess the...

Hello, i'm having trouble evaluating my gamma factor for my special relativity homework, because I need to compute 1 minus a very small number (8.57*10^-13). My calculator treats this value as simply 1, as does Mathematica. Although I don't know much about it, and maybe there's a way to force...