[proFit-list] Finding the min or max of data
Davide Guarisco
guarisco at sbcglobal.net
Fri May 10 23:40:14 CDT 2013
On May 10, 2013, at 9:45 AM, Chris Lee wrote:
> If you are comparing exact values then the equality should always work. I agree that if you were to compute two values using different function calls then comparing for equality is a bad idea, but, if a function reads the same value from a table twice and an equality test fails, I would be worried.
But here you would be comparing a data cell from the table with a result of a function (either pro Fit's built-in Statistics or a numpy function). You don't know a priori what those functions do. In decades of programming I have learned the hard way to be conservative and practice "defensive programming".
>
> I don't think that the pro fit compiler optimizes all that well, so I would expect that internal code that loops is faster than compiled pascal that loops. Likewise, the numpy.argmax() function must loop internally, but it is amazingly fast, and no matter how long an array is, I've never observed a delay in the function call.
My experience tells me that you are correct in that the native pro Fit functions execute faster than compiled pro Fit programs. Similarly, numpy is known to be fast, almost as fast as compiled code, but not quite as fast.
Davide
> On 10 May 2013 (W: 19), at 16:40, Davide Guarisco <guarisco at sbcglobal.net> wrote:
>
>>
>> On May 10, 2013, at 1:41 AM, Chris Lee wrote:
>>
>>> Why not call the statistics function first to determine the min and max. Then, in the loop, check for equality?
>>
>>
>> Two reasons:
>> 1. With floating point numbers, equality check is always tricky
>> 2. The Statistics function also will have to loop through the data to find min and max. It should be more efficient to loop through the data once. This assumes that the compiled proFit programs and "native" Statistics functions run equally fast. That may not be the case.
>>
>> Davide
>>
>>
>>
>>
>>
>>>
>>> I am responsible for the concept of this message. Autocorrect is responsible for the contents.
>>>
>>> On 10/05/2013, at 5:15 AM, Davide Guarisco <guarisco at sbcglobal.net> wrote:
>>>
>>>> Then you have never dealt with large amount of data. I do, and I can assure you that pro Fit takes an awfully long time to sort a substantial data file (when it doesn't hang).
>>>>
>>>> But I happen to have a fondness for good and efficient programming, and sorting really isn't the way to solve this problem. The best sorting algorithms behave as O(n log n), whereas this problem can be solved in O(n).
>>>>
>>>>
>>>> Consider this simple code:
>>>>
>>>> procedure findminmax(var xmin,xmax: extended;var imin,imax: integer);
>>>> {Finds the minimum (xmin), maximum (xmax) values and their indices of the Y column of the current data window}
>>>> var ymin,ymax,yx: extended;
>>>> i: integer;
>>>>
>>>> begin
>>>> ymin:=InvalidNum;
>>>> ymax:=InvalidNum;
>>>> imin:=InvalidNum;
>>>> imax:=InvalidNum;
>>>> {Find first valid data}
>>>> for i:=1 to NrRows do
>>>> if not(invalid(data[i,YColumn])) then
>>>> begin
>>>> ymin:=data[i,YColumn];
>>>> ymax:=ymin;
>>>> imin:=i;
>>>> imax:=imin;
>>>> Leave;
>>>> end;
>>>> for i:=i+1 to NrRows do
>>>> begin
>>>> if not(invalid(data[i,YColumn])) then
>>>> begin
>>>> yx:=data[i,YColumn];
>>>> if yx>ymax then
>>>> begin
>>>> ymax:=yx;
>>>> imax:=i
>>>> end
>>>> else
>>>> if yx<ymin then
>>>> begin
>>>> ymin:=yx;
>>>> imin:=i
>>>> end;
>>>> end;
>>>> end;
>>>> xmin:=ymin;
>>>> xmax:=ymax
>>>> end; {findminmax}
>>>>
>>>> I think this does what the OP asked for.
>>>>
>>>> Davide
>>>>
>>>>
>>>>
>>>>
>>>> On May 9, 2013, at 8:19 AM, Camaioni, Donald M wrote:
>>>>
>>>>> Hi Davide,
>>>>>
>>>>> It can be programmed via a single command using pro Fit's built in sort routine, e.g., in pro Fit's Pascal programming language:
>>>>>
>>>>> Sort(window , referenceCol , order , selectionOnly );
>>>>>
>>>>> Efficiency of this routine has never been an issue for me.
>>>>>
>>>>> Don
>>>>>
>>>>> On May 9, 2013, at 7:44 AM, Davide Guarisco wrote:
>>>>>
>>>>> I would advise against doing that. Sorting is extremely inefficient.
>>>>>
>>>>> Davide
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On May 8, 2013, at 7:15 AM, Camaioni, Donald M wrote:
>>>>>
>>>>> I am thinking you would just sort the data and read values from either the resulting first or last row. -Don
>>>>>
>>>>> On May 8, 2013, at 6:54 AM, Dave wrote:
>>>>>
>>>>> Hi folks,
>>>>>
>>>>> While I am anxiously awaiting the Cocoa based, Retina- ready version of pro Fit, a question:
>>>>>
>>>>> Is there an easy way to find the local minima/ maxima of a set of data or selection of data, and report that point? I mean finding the max or min y value and reporting the x,y coördinate.
>>>>>
>>>>> Thanks,
>>>>> Dave
>>>>> _______________________________________________
>>>>> proFit-list mailing list
>>>>> proFit-list at quantum-soft.com<mailto:proFit-list at quantum-soft.com>
>>>>> http://quantum-soft.com/mailman/listinfo/profit-list_quantum-soft.com
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> proFit-list mailing list
>>>>> proFit-list at quantum-soft.com<mailto:proFit-list at quantum-soft.com>
>>>>> http://quantum-soft.com/mailman/listinfo/profit-list_quantum-soft.com
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> proFit-list mailing list
>>>>> proFit-list at quantum-soft.com<mailto:proFit-list at quantum-soft.com>
>>>>> http://quantum-soft.com/mailman/listinfo/profit-list_quantum-soft.com
>>>>>
>>>>>
>>>>> ––––––––––––––––––––––––––––––––
>>>>> Donald Camaioni, PhD
>>>>> Staff Scientist
>>>>> Catalysis Science Group
>>>>> PHYSICAL SCIENCES DIVISION
>>>>>
>>>>> Pacific Northwest National Laboratory
>>>>> 902 Battelle Boulevard
>>>>> P.O. Box 999, MSIN K2-57
>>>>> Richland, WA 99352 USA
>>>>>
>>>>> Tel: (509) 375-2739
>>>>> donald.camaioni at pnnl.gov<mailto:donald.camaioni at pnnl.gov>
>>>>> www.pnl.gov<http://www.pnl.gov/>
>>>>
>>>> Davide
>>>> --
>>>> Davide Guarisco
>>>> 5143 Halifax Dr.
>>>> San José, CA 95130
>>>> Home: (408) 376-0898
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> proFit-list mailing list
>>>> proFit-list at quantum-soft.com
>>>> http://quantum-soft.com/mailman/listinfo/profit-list_quantum-soft.com
>>>
>>> _______________________________________________
>>> proFit-list mailing list
>>> proFit-list at quantum-soft.com
>>> http://quantum-soft.com/mailman/listinfo/profit-list_quantum-soft.com
>>
>> Davide
>> --
>> Davide Guarisco
>> 5143 Halifax Dr.
>> San José, CA 95130
>> Home: (408) 376-0898
>>
>>
>>
>>
>> _______________________________________________
>> proFit-list mailing list
>> proFit-list at quantum-soft.com
>> http://quantum-soft.com/mailman/listinfo/profit-list_quantum-soft.com
>
>
> _______________________________________________
> proFit-list mailing list
> proFit-list at quantum-soft.com
> http://quantum-soft.com/mailman/listinfo/profit-list_quantum-soft.com
Davide
--
Davide Guarisco
5143 Halifax Dr.
San José, CA 95130
Home: (408) 376-0898
More information about the proFit-list
mailing list