The Rise of Meta Learning
- This evolution from block orientation to solving a rubik’s cube is fueled by a Meta-Learning algorithm controlling the training data distribution in simulation, Automatic Domain Randomization (ADR).
- Although not as surprising as misclassifications do to hardly noticeable adversarial noise injections, Deep Convolutional Neural Networks will not generalize when trained on images in simulation (displayed below on the left) to real visual data (shown below on the right) without special modifications.
- Domain Randomization appears to be the key to bridging the Sim2Real gap, allowing Deep Neural Networks to generalize to real data when trained on simulation.
- It seems likely that the ideas of Automatic Domain Randomization would be improved with advanced search algorithms, i.e. something like the Population-Based Search proven useful in Data Augmentation by Researchers at UC Berkeley or AutoAugment from Google.
- Most data augmentation searches (even automatic domain randomization) is constrained to a set of transformations available to the meta-learning controller.
- The problem with this approach is that if we cannot distinguish between human and machine intelligence, it doesn’t necessarily mean that they are the same kind of intelligence.
- The researchers discovered that according to the data, patients who suffered from pneumonia and were asthmatic had a higher recovery rate than the others.
- It goes to prove two things; first of all, that human intuition is essential since the doctors immediately flagged this issue when confronted with the results of the automated system.
- Even though automation is inevitable, we have to take into consideration the human element and create AI systems which are sensitive to our needs.
- In fact, according to media reports, it seems like over 12 million people were affected with travel bans, as punishment for their behaviour so far.
On Mind Uploads
- One interesting consequence of using WBE is the creation of Artificial General Intelligence (AGI).
- Since computations on computers are carried out orders of magnitude faster than computations in the human brain, a WBE mind might think a lot faster than humans; in that sense, it might make more sense to talk about an Artificial Superintelligence.
- Destructive WBE uploads your mental state and destroys your original mental substrate in the process.
- You see, our mental states (including cognition, memory, consciousness, etc.) are a result of (a lot of) computations, and computations are substrate-independent (meaning they can be performed on different kinds of substrates).
- In other words, you can’t tell the difference between being run on flesh (your brain) or on a computer — which is why, yes, you (meaning the “original” you) live on after your mind upload.
Nonlinear Dynamics and Chaos
- This post is a summary of the theory of nonlinear dynamics and chaos that I have recently learned from an online course by the Santa Fe Institute.
- Meanwhile for the second logistic map (R = 3.2), we see that x eventually oscillates between two values, which explains why the blue dot in the previous chart perpetually moved between two locations.
- Remember from the x(t) vs x(t+1) chart that the value of R determines how the dynamics converge over time.
- In this section, we will plot R vs the number of stable points that the system settles down to (a.k.a. attractors).
- Two points, one starting at 0.20000 and the other 0.20001 ended up in a completely different trajectory when R = 4 (referring to the bifurcation diagram, we see that this leads to a huge number of attractors).
Making Convolutional Networks Shift-Invariant Again
- Existing work propose various methods to add anti-aliasing capabilities to downsampling, such as by extracting features densely, by softly gating between max and average pooling or with convolutional kernel networks.
- By computing the internal feature distance throughout the layers of the VGG network for various horizontal and vertical shifts, we observe that the anti-aliased network maintains better shift-equivariance, and the resulting output is more shift-variant.
- For image generation, low-pass filters are added to strided-convolution layers of the U-Net architecture.
- Classical signal processing practices blurring for anti-aliasing, but it has disappeared in modern convolutional networks in an aim to optimize performance.
- With this paper's results demonstrating adding anti-aliasing to current convolutional networks improves both shift-equivariance and task performance, it is perhaps time to bring back blurring filters and make convolutional networks shift-invariant again!
Deep Sea Correlation
- For our features, we’re going to start with predicting Depth versus temperature, to see what we have there.
- In order to validate our Regression models, as well as our baseline, we will use mean absolute error in tandem with R².
- I chose to then investigate the mean of the TestX, and TestY, given that there is so much data here, I might consider going back and adjusting for a bigger train set, and smaller test set, because the test is sitting at about 250,000 observations, which is enormous, and just makes validation, and model exploration impossible to work with.
- So looking at this, it’s pretty obvious that we need a different model to predict this.
- We also see the reason why we had such a low mean absolute error, while also having such a low R² score, the model was predicting negatives.
Let’s Get Back to Basics
- So it’s no surprise that there is a strong trend in the market to become data driven, in part because cloud technologies and the growing usage of SaaS make data more accessible than ever.
- I’m also seeing that while there’s more appetite than ever to be data-driven, it’s harder than ever to be good at it.
- In fact, I’d say there’s actually an opposite trend around the fundamentals of data (accuracy, quality, reliability, governance, access, protection, security) — these are all extremely challenging to get right, and getting worse to manage.
- Why is it that our standard for reliability is so high for everything we do in business — it’s part of what encourages this data driven trend in the first place!
Data Science and Healthcare
- I recently went to a Data & Healthcare meetup to see how a renowned cancer research institute, Memorial Sloan Kettering Cancer Center (MSK), applies and uses Data Science.
- It was very inspiring to hear that MSK has “a lot of data” and they’re exploring how Data Science can be used to be beneficial and impactful to provide the best patient centered experience.
- Another speaker from MSK stated that some providers don’t like to click on checkboxes to document their notes and that they prefer to write it out in prose.
- While I don’t believe that quote was necessarily addressed towards Data Science, it is inspiring and interesting to see how a company such as MSK promotes and shares their interest and excitement in the field of Data Science and analytics to ultimately provide the best patient-centered care.
Build Face Recognition as a REST API
- This post will demonstrate how to build a RESTful API for face_recognition on Linux servers using Python Flask.
- The second function face_rec check if the face in the image file is a known face in the dataset, and return the name of the person.
- It takes two image files from form data of a POST request, and calls compare_faces to check if they are a match, then returns a JSON format result.
- The second API is face_rec, which takes one image file as input and calls face_rec to check if it is a known face, then returns the name of the person in JSON format.
- You can add more features to the API, for instance, to return face locations and facial features so that you can implement the same things on mobile devices as you can do on Linux servers.
Real-time inference at scale on AWS
- Running machine learning in production is not just about deploying models, but also ensuring your deployments are maintainable, your spend is optimized, and your web services are scalable.
- An increase in traffic to service B will be scheduled on much cheaper CPU resources without requiring additional GPUs. In addition to choosing the appropriate instance types for your inference workload, the simplest way to optimize spending is to automatically adjust the size of the cluster based on the aggregate resource requests of all containers on the cluster.
- Although inference is often made more efficient by utilizing GPU infrastructure, it may be wasteful to run pre-processing and post-processing code on GPUs. Creating a Kubernetes cluster that runs on top of CPU and GPU Auto Scaling Groups helps address this problem.