Skip to main content

harry palmer

harry palmer's Public Library

  • Python

     
    Works with: Python version 3.x

     

    Both implementations use the deque datatype. 

     

  • You can calculate a running mean with:

      
    import numpy as np  def runningMean(x, N):     y = np.zeros((len(x),))     for ctr in range(len(x)):          y[ctr] = np.sum(x[ctr:(ctr+N)])     return y/N
      

    But it's slow.

      

    Fortunately, numpy includes a convolve function which we can use to speed things up. The running mean is equivalent to convolving x with a vector that is N long, with all members equal to 1/N. The numpy implementation of convolve includes the starting transient, so you have to remove the first N-1 points:

      
    def runningMeanFast(x, N):     return np.convolve(x, np.ones((N,))/N)[(N-1):]
      

    On my machine, the fast version is 20-30 times faster, depending on the length of the input vector and size of the averaging window.

      

    Note that convolve does include a 'same' mode which seems like it should address the starting transient issue, but it splits it between the beginning and end.

  • Convolution is much better than straightforward approach, but (I guess) it uses FFT and thus quite slow. However specially for computing the running mean the following approach works fine

      
    def running_mean(x, N):     cumsum = numpy.cumsum(numpy.insert(x, 0, 0))      return (cumsum[N:] - cumsum[:-N]) / N 
      

    The code to check

      
    In[3]: x = numpy.random.random(100000) In[4]: N = 1000 In[5]: %timeit result1 = numpy.convolve(x, numpy.ones((N,))/N, mode='valid') 10 loops, best of 3: 41.4 ms per loop In[6]: %timeit result2 = running_mean(x, N) 1000 loops, best of 3: 1.04 ms per loop
      

    Note that numpy.allclose(result1, result2) is True, two methods are equivalent. The greater N, the greater difference in time.

  • Best way to apply a moving/sliding average (or any other sliding window function) to a signal is by using numpy.convolve().

      
    def movingaverage(interval, window_size):     window = numpy.ones(int(window_size))/float(window_size)     return numpy.convolve(interval, window, 'same')
      

    Here, interval is your x array, and window_size is the number of samples to consider. The window will be centered on each sample, so it takes samples before and after the current sample in order to calculate the average. Your code would become:

      
    plot(x,y) xlim(0,1000)  x_av = movingaverage(interval, r) plot(x_av, y)  xlabel("Months since Jan 1749.") ylabel("No. of Sun spots") show()
      

    Hope this helps!

  • import numpy as np  dataset = [1,5,7,2,6,7,8,2,2,7,8,3,7,3,7,3,15,6]  def movingaverage(values,window):  weigths = np.repeat(1.0, window)/window  #including valid will REQUIRE there to be enough datapoints.  #for example, if you take out valid, it will start @ point one,  #not having any prior points, so itll be 1+0+0 = 1 /3 = .3333  smas = np.convolve(values, weigths, 'valid')  return smas # as a numpy array  #Will print out a 3MA for our dataset print movingaverage(dataset,3)

  • Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar interface to query data living in other data storage systems.

  • There’s actually nothing to see because array indexing goes straight into the NumPy C extensions, and the Python profiler can’t see what’s going on there. (The visualizations were made with SnakeViz.)

     

    With this in mind I rewrote my function (and its supporting code) so that during the loop all the data would be in plain NumPy arrays. The inputs are DataFrames and Series, which I reorganize into arrays and scalars. At the end I transform the array of weights into a Series with the appropriate index. Here are the before and after of the module I was working on, as well as the diff. (The household_weights function is the high level entry point.)

     

    I should note that using Pandas is fast enough most of the time, and you get the benefit of Pandas’ sophisticated indexing features. It’s only in loops that the microseconds start to add up to minutes.

  • When you're building a graph, table, or other visual display with D3.js, it requires a different kind of thinking. Many developers are used to very imperative styles ("Here's some data. Do X with it!") D3.js has more of a declarative or functional style ("For all of the rows in the table, here's what should be done with them.") 

    Declarative techniques work really well, but they can be a daunting change of pace. So, keep in mind:​ You don't iterate over data structures and do things. Instead, you select sets of items. Those items may not even yet exist in your document at the time you select them. But it doesn't matter! You're declaring a set, and then what should be done to represent that set. If the set is empty (intially), the next move is probably a .enter().append("..."). In other words for-this-set.lets-create-some-elements().here-they-are("...")

    Understanding this declarative/functional model goes a long way in getting your D3.js projects off the ground.

    • Imperative vs Declarative

       
      02 April 2013
       

      Let's generalize and say that there are two ways in which we can write code: imperative and declarative.

        

      We could define the difference as follows:

        
         
      • Imperative programming: telling the "machine" how to do something, and as a result what you want to happen will happen.
      •  
      • Declarative programming: telling the "machine"1 what you would like to happen, and let the computer figure out how to do it.
      •  
        

      1 Computer/database/programming language/etc

    • Functional Programming in Javascript

       

      Functional programming provides developers with the tools to abstract common collection operations into reusable, composable building blocks. You'll be surprised to learn that most of the operations you perform on collections can be accomplished with five simple functions:

       
         
      1. map
      2.  
      3. filter
      4.  
      5. mergeAll
      6.  
      7. reduce
      8.  
      9. zip
      10.  
       

      Here's my promise to you: if you learn these 5 functions your code will become shorter, more self-descriptive, and more durable. Also, for reasons that might not be obvious right now, you'll learn that these five functions hold the key to simplifying asynchronous programming. Once you've finished this tutorial you'll also have all the tools you need to easily avoid race conditions, propagate and handle asynchronous errors, and sequence events and AJAX requests. In short, these 5 functions will probably be the most powerful, flexible, and useful functions you'll ever learn.

  • up vote  12  down vote    favorite 
      
           
     
      

    I'm looking for a definitive answer to why extending built-in prototypes is so heavily chastised in the JS developer community. I've been using the Prototype JS framework for a while, and to me doing [1,2,3].each(doStuff) seems much more elegant than $.each([1,2,3], doStuff). I know that it creates "namespace pollution," but I stil don't understand why it's considered to be a bad thing. Also is there any real performance degradation associated with extending built-in prototypes? Thanks!

  • I recently wrote a post about my Javascript Prototypal Object Oriented Enlightenment.  In that post I discussed how I learned to break my Classical Object Oriented habits and stop forcing Javascript to do something it really wasn't suited to do. I focused more on my process of transitioning from failed Class-based OO attempts in Javascript to Prototypal Enlightenment and the differences between the two approaches.

      

    I wanted to follow up that post with a post focused on the implementation and use of Prototypal OO in Javascript. We'll start with the short function that inspired the last post and allows for easier Prototypal OO in Javascript...

  • JavaScript typed arrays are array-like objects and provide a mechanism for accessing raw binary data. As you already know, Array objects grow and shrink dynamically and can have any JavaScript value. JavaScript engines perform optimizations so that these arrays are fast. However, as web applications become more and more powerful, adding features such as audio and video manipulation, access to raw data using WebSockets, and so forth, it has become clear that there are times when it would be helpful for JavaScript code to be able to quickly and easily manipulate raw binary data in typed arrays.

    • There are several different ways to represent floating-point numbers in computers: most architectures now use the IEEE754 standards, representing double precision numbers with 64 bits (a.k.a double, or float64) and single precision numbers with 32 bits (a.k.a float32). As its name suggests, a float64 has more precision than a float32, so it’s generally advised to use it, unless you are in a performance-sensitive context. Using float32 has the following advantages:

       
         
      • Float32 operations often require less CPU cycles as they need less precision.
      •  
      • There is additional CPU overhead if the operands to a float64 operation are float32, since they must first be converted to float64. (This can be the case in code that uses Float32Array to store data, which is often true for JS code using WebGL, as well as Emscripten-compiled code where a float in C++ is stored as a float32 in memory but uses JavaScript float64 arithmetic.)
      •  
      • Float32 numbers take half the space, which can decrease the total memory used by the application and increase the speed of memory-bound computations.
      •  
      • Float32 specializations of math functions in several standard C libraries we’ve tested are way faster than their float64 equivalents.
      •  
       

      In JavaScript, number values are defined to be float64 and all number arithmetic is defined to use float64 arithmetic. Now, one useful property of float64 arithmetic that JavaScript engines have taken advantage of for a long time is that, when a float64 is a small-enough integer, the result of a float64 operation is the same as the corresponding integer operation when there is no integer overflow. JavaScript engines take advantage of this by representing integer-valued numbers as raw integers and using integer arithmetic (and checking for overflow). In fact, there are at least 3 different integer representations in use that I know of: 31-bit integers, int32_t and

  • When You Can't Count On Your Numbers

      
     
     
    By YUI TeamMarch 10, 2009
      
     

    JavaScript has a single number type: IEEE 754 Double Precision floating point. Having a single number type is one of JavaScript's best features. Multiple number types can be a source of complexity, confusion, and error. A single type is simplifying and stabilizing.

    Unfortunately, a binary floating point type has some significant disadvantages. The worst is that it cannot accurately represent decimal fractions, which is a big problem because humanity has been doing commerce in decimals for a long, long time. There would be advantages to switching to a binary-based number system, but that is not going to happen. As a consequence, 0.1 + 0.2 === 0.3 is false, which is the source of a lot of confusion.

    When working with floating point numbers, it is important to understand the limitations and program defensively. For example, the Associative Law does not hold. (((a + b) + c) + d) is not guaranteed to produce the same result as ((a + b) + (c + d)).

    Let's demonstrate this. We'll start with a partial_reduce function. We pass it an array and a function, and it returns in array containing the results of calling the function on pairs of elements. This sort of thing might be popular in the future to take advantage of parallelism because work on each of the pairs could happen simultaneously.

  • Exercise 6: Strings and Text

     

    While you have been writing strings, you still do not know what they do. In this exercise we create a bunch of variables with complex strings so you can see what they are for. First an explanation of strings.

     

    A string is usually a bit of text you want to display to someone, or "export" out of the program you are writing. Python knows you want something to be a string when you put either " (double-quotes) or ' (single-quotes) around the text. You saw this many times with your use of print when you put the text you want to go inside the string inside " or ' after the print to print the string.

     

    Strings may contain the format characters you have discovered so far. You simply put the formatted variables in the string, and then a % (percent) character, followed by the variable. The only catch is that if you want multiple formats in your string to print multiple variables, you need to put them inside ( ) (parenthesis) separated by , (commas). It's as if you were telling me to buy you a list of items from the store and you said, "I want milk, eggs, bread, and soup." Only as a programmer we say, "(milk, eggs, bread, soup)."

  • Strings are amongst the most popular types in Python. We can create them simply by enclosing characters in quotes. Python treats single quotes the same as double quotes.

     

    Creating strings is as simple as assigning a value to a variable. For example:

     
    var1 = 'Hello World!' var2 = "Python Programming"
     

    Accessing Values in Strings:

     

    Python does not support a character type; these are treated as strings of length one, thus also considered a substring.

     

    To access substrings, use the square brackets for slicing along with the index or indices to obtain your substring. Following is a simple example:

  • Default Parameters in JavaScript

       

      Originally published in the   A Drip of JavaScript newsletter

        

    Some languages — like Ruby, CoffeeScript, and upcoming versions of JavaScript — have the ability to declare default parameters when defining a function. It works like this:

     
    function myFunc(param1, param2 = "second string") {     console.log(param1, param2); }  // Outputs: "first string" and "second string" myFunc("first string");  // Outputs: "first string" and "second string version 2" myFunc("first string", "second string version 2"); 
     

    Unfortunately, this construct isn't available in current versions of JavaScript. So what can we do to achieve this behavior with our current set of tools?

      

    The simplest solution looks like this:

  • In 1872 two men began work on a lexicon of words of Asian origin used by the British in India. Since its publication the 1,000-page dictionary has never been out of print and a new edition is due out next year. What accounts for its enduring appeal?

     

    Hobson-Jobson is the dictionary's short and mysterious title.

     

    The subtitle reveals more: "A glossary of colloquial Anglo-Indian words and phrases, and of kindred terms etymological, historical, geographical and discursive. By Colonel Henry Yule and AC Burnell."

     

    But even the word "discursive" doesn't quite prepare the reader for what is to come.

  • Introduction

      

    Surprisingly, a topic of named function expressions doesn’t seem to be covered well enough on the web. This is probably why there are so many misconceptions floating around. In this article, I’ll try to summarize both — theoretical and practical aspects of these wonderful Javascript constructs; the good, bad and ugly parts of them.

      

    In a nutshell, named function expressions are useful for one thing only — descriptive function names in debuggers and profilers. Well, there is also a possibility of using function names for recursion, but you will soon see that this is often impractical nowadays. If you don’t care about debugging experience, you have nothing to worry about. Otherwise, read on to see some of the cross-browser glitches you would have to deal with and tips on how work around them.

      

    I’ll start with a general explanation of what function expressions are how modern debuggers handle them. Feel free to skip to a final solution, which explains how to use these constructs safely.

  • Published on Oct 3, 2013

    http://jamiedreyer.com/elbow-tendonit... I show you the exact techniques I used to heal my own painful tendonitis as well as numerous clients. If you've tried stretching with no luck, that's because stretching is part of the problem.

1 - 20 of 8236 Next › Last »
20 items/page

Highlighter, Sticky notes, Tagging, Groups and Network: integrated suite dramatically boosting research productivity. Learn more »

Join Diigo