Center for Human Machine Collaboration (CHMC)
Is there a fundamental basis for human intelligence? In the Center for Human-Machine Collaboration (CHMC) we are exploring this question through research that is focused on understanding the brain’s capability to learn, recall, and adapt to uncertainty. We apply our research by creating algorithms, tools, and systems to enhance learning, decision making, and intelligent behavior in machines.
Research
Cognitive Teaming Systems
Cognitive Teaming Systems
- Human-Machine Symbiosis
- Team Neurosynchrony
- Cognitive Processing for Autonomous Systems
Current and Recent Projects
Cognitive Processing for Adaptive Decision Making
Autonomy is the key to growth and scalability of complex systems. Cognitive systems can supply the two key ingredients for this: 1) The ability to orient; that is, the ability to synthesize information observed and convert it into information that can be acted upon, creating a coherent manipulable view of the world, and 2) The ability to decide; that is, weight alternate options and select. Our objective is to research an approach to cognitive processing with a focus on hybrid technologies to create leverage toward achieving autonomous systems. The approach we developed utilizes inspiration from the brain: specifically, how the brain stores experiences in memory, generalizes semantics from memory, and reasons over it to produce decisions.
This work is funded through internal research funds.
Decision Making and Conceptual Structure
Decision Making and Conceptual Structure
- Neurocognitive Models of Sensemaking
- Knowledge Representation
Current and Recent Projects
Integrated Cognitive Architectures for Understanding Sensemaking (ICArUS)
Sensemaking is a proactive form of situation awareness where noisy, sparse, and uncertain multimodal information is synthesized into explanatory and predictive hypotheses. Several collaborative modeling efforts spanned cognitive architectures using production systems and detailed neural models designed to reveal the neural principles and cognitive tradeoffs that examine human reasoning, decision-making, and cognitive bias.
Supported by the Intelligence Advanced Research Projects Activity (IARPA) via Department of the Interior (DOI) contract number D10PC20021 from December 2010 to November 2014.
Knowledge Represenation in Neural Systems (KRNS)
The KRNS program challenges researchers to provide and evaluate novel theories for knowledge representation by demonstrating their utility in interpreting neural activity to decode the concepts that generated it. In collaborations with several universities, we conducted research in widely varying semantic models as candidate representations for the brain, new spatiotemporal resolutions in fMRI BOLD neuroimaging to examine dynamic activity, and machine learning methods to extract the underlying neural representations of concepts.
Supported by the Intelligence Advanced Research Projects Activity (IARPA) via AFRL contract number FA8650-13-C-7356 from September 2013 to January 2015.
Neuroscience of Learning and Memory
Neuroscience of Learning and Memory
- Noninvasive, Intracranial, Neurostimulation and Recording
- Memory Enhancement
- Enhanced Learning of Complex Skills
- Understanding Basis of Expertise
- Memory Consolidation
- Personalized Predictive Models of Behavior
Current and Recent Projects
Restoring Active Memory: Replay (RAM-REPLAY)
HRL is leading a team of 5 universities and 5 consultants to develop a man-portable, model-driven tCS technology that can accelerate skill acquisition and specific memory consolidation by closed-loop intervention during both waking and sleep. (DARPA BTO #W911NF-16-C-0018)
Transcranial Stimulation To Improve Memory (tSTIM)
HRL, with subcontractor McGill University, investigated the use of non-invasive stimulation to modulate single units and LFPs across multiple brain areas and thereby enhance memory function, in direct comparison with invasive stimulation methods, and also developed a predictive computational model of stimulation-enhanced short-term memory behavior. (DARPA BTO #N66001-14-C-4066)
Memory Enhancement by Modulation of Encoding Strength (MEMES)
HRL, with subcontractor McGill University, is investigating the effects of closed-loop multi-site non-invasive stimulation on long-term memory behavior and underlying neurophysiology in two neocortical areas and hippocampus. (DARPA BTO # N66001-16-C-4058)
Neurostimulation
Training methods to reach expert level performance is time-consuming and costly, while high physical/cognitive performance and skill mastery in workers/soldiers are in great demand. We strive to radically improve training with the use of novel neurostimulation approaches through an understanding of the neural basis of expertise in order to enable expert level performance for large fractions of the population by improving endogenous physical and cognitive processes. Our objective is to develop fundamental theories and techniques for neurostimulative/neuromodulatory interventions for cognitive and physical performance enhancement.
We are also looking into automated synthesis of software for complex cryptographic protocols.
People
Rajan Bhattacharyya
Jaehoon Choe
Vincent De Sapio
Mike Howard
Nick Ketz
Iman Mohammadrezazadeh
Praveen Pilly
Steven Skorheim
Kenji Yamada
eCortex, Inc.
Prof. Mark Schnitzer, Stanford University
Drs. Christian Lebiere, Prof. John R. Anderson and James Staszeqski, Carnegie Mellon University
Dr. Randall O'Reilly, University of Colorado Boulder
Profs. Stephen Grossberg and Gail Carpenter, Boston University
Prof. Jeff Krichmar, University of California Irvine
Prof. Giorgio Ascoli, George Mason University
Professor Vince Clark, University of New Mexico
Professor Chris Pack, McGill University
Publications
Click to show/hide the list of Papers
| Authors | Title | Publication | Year |
|---|---|---|---|
| Ketz, N. A., Jones, A. P., Bryant, N. B., Clark, V. P., and Pilly, P. K. | Closed-loop slow-wave tACS improves sleep dependent long-term memory generalization by modulating endogenous oscillations | Journal of Neuroscience, 38(33), 7314-7326 | 2018 |
| Pilly, P. K., Howard, M. D., and Bhattacharyya, R. | Modeling contextual modulation of memory associations in the hippocampus | Frontiers in Human Neuroscience, 12, 442 | 2018 |
| Robinson, C. S. H., Bryant, N. B., Maxwell, J. W., Jones, A. P., Robert, B., Lamphere, M., Combs, A., Al Azzawi, H. M., Gibson, B. C., Sanguinetti, J. L., Ketz, N. A., Pilly, P. K., and Clark, V. P. | The benefits of closed-loop transcranial alternating current stimulation on subjective sleep quality | Brain Sciences, 8(12), 204 | 2018 |
| Jones, A. P., Choe, J., Bryant, N. B., Robinson, C. S. H., Ketz, N. A., Skorheim, S. W., Combs, A., Lamphere, M. L., Robert, B., Gill, H. A., Heinrich, M. D., Howard, M. D., Clark, V. P., and Pilly, P. K. | Dose-dependent effects of closed-loop tACS delivered during slow-wave oscillations on memory consolidation | Frontiers in Neuroscience: Neural Technology, 12, 867 | 2018 |
| Lerner, I., Ketz, N. A., Jones, A. P., Bryant, N. B., Robert, B., Skorheim, S. W., Hartholt, A., Rizzo, A. S., Gluck, M. A., Clark, V. P., and Pilly, P. K. | Transcranial current stimulation during sleep facilitates insight into temporal rules, but does not consolidate memories of individual sequential experiences | Scientific Reports, in press | 2018 |
| Patel, A. N., Howard, M. D., Roach, S. M., Jones, A. P., Bryant, N. B., Robinson, C. S. H., Clark, V. P., and Pilly, P. K. | Mental state assessment and validation using personalized physiological biomarkers | Frontiers in Human Neuroscience, 12, 221 | June 2018 |
| Ryan J. Hubbard, Nicholas A. Ketz, Aaron P. Jones, Bradley Robert, Natalie B. Bryant, Steven W. Skorheim, Shane Roach, Vincent P. Clark, Praveen K. Pilly | Memory Reactivation with Neurostimulation during Sleep Elicits Electrophysiological Responses that Predict Behavioral Changes | Presented at the 25th Annual Meeting of the Cognitive Neuroscience Society | March 2018 |
| S. M. Salas, R. P. Patrick, S. Roach, M. E. Phillips, N. D. Stepp, J. Cruz-Albrecht, V. De Sapio, T-C. Lu, V. Sritapan | Neuromorphic and Early Warning Behavior- Based Authentication in Common Theft Scenarios | In Proceedings of the 2017 IEEE International Symposium on Technologies for Homeland Security (HST) | April 2017 |
| Krause, M. R., Zanos, T. P., Csorba, B. A., Pilly, P. K., Choe, J., Phillips, M. E., Pack, C. C. | Transcranial direct current stimulation facilitates associative learning and alters functional connectivity in the primate brain | Current Biology, 27(20), 3086-3096 | 2017 |
| M. E. Phillips, N. D. Stepp, J. Cruz-Albrecht, V. De Sapio, T-C. Lu, and V Sritapan | Neuromorphic and early warning behavior-based authentication for mobile devices | In Proceedings of the 2016 IEEE International Symposium on Technologies for Homeland Security (HST) | May 2016 |
| V. De Sapio, M. Howard, D. Korchev, R. Green, R. Gardner, and L. Bruchal | Demographic specific musculoskeletal models of factory worker performance, fatigue, and injury | In Proceedings of the 2016 IEEE International Aerospace Conference | March 2016 |
| Datta, M. R. Krause, P. K. Pilly, J. Choe, T. P. Zanos, C. Thomas, and C. C. Pack | On comparing in vivo intracranial recordings in non-human primates to predictions of optimized transcranial electrical stimulation | Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Orlando, FL. | 2016 |
| Choe, J., Coffman, B. A., Bergstedt, D. T., Ziegler, M. D., Phillips, M. E. | Transcranial Direct Current Stimulation Modulates Neuronal Activity and Learning in Pilot Training | Frontiers in human neuroscience, 10 | 2016 |
| Phillips, M. E., Stepp, N. D., Cruz-Albrecht, J., De Sapio, V., Lu, T. C., Sritapan, V. | Neuromorphic and Early Warning Behavior-based Authentication for Mobile Devices | Technologies for Homeland Security (HST), 2016 IEEE Symposium on (pp. 1-5) IEEE | May 2016 |
| V. De Sapio, M. Howard, D. Korchev, R. Green, R. Gardner, L. Bruchal | Demographic Specific Musculoskeletal Models of Factory Worker Performance, Fatigue, and Injury | Proceedings of IEEE Aerospace Conference | March 2016 |
| Bhattacharyya R, Coffman BA, Choe J, Phillips ME | Human Augmentation: Does Neurotechnology Make a Better Brain? | IEEE Computer | 2016 |
| Vu, A. T., Phillips, J. S., Kay, K., Phillips, M. E., Johnson, M. R., Shinkareva, S. V., Bhattacharyya, R. | Using Precise Word Timing Information Improves Decoding Accuracy in a Multiband-accelerated Multimodal Reading Experiment | Cognitive Neuropsychology, 33(3-4), 265-275 | 2016 |
| Craig, A. B., Phillips, M. E., Zaldivar, A., Bhattacharyya, R., Krichmar, J. L. | Investigation of Biases and Compensatory Strategies Using a Probabilistic Variant of the Wisconsin Card Sorting Test | Frontiers in psychology, 7 | 2016 |
| Sun, Y., O’Reilly, R. C., Bhattacharyya, R., Smith, J. W., Liu, X., Wang, H. | Latent Structure in Random Sequences Drives neural learning toward a rational bias | Proceedings of the National Academy of Sciences, 112(12), 3788-3792 | 2015 |
| Phillips ME, Phillips J, Tubridy SM, Johnson MR, Shinkareva SV, Millin R, Vu AT, Yacoub E, Gureckis T, Grossman M, Bhattacharyya R | A neurosemantic behavioral feature model predicts conceptual representation in the brain | Cognitive Neuroscience Society 2015 (Abstract accepted for full paper submission to Special Issue of Cognitive Neuropsychology) | 2015 |
| Ni, K., Benvenuto, J., Bhattacharyya, R., Millin, R. | Feature Transformation of Neural Activity with Sparse and Low-Rank Decomposition | Accepted for SPIE Proceedings 9417, Medical Imaging 2015: Biomedical Applications in Molecular, Structural, and Functional Imaging | 2015 |
| Vu AT, Phillips J, Kay K, Phillips ME, Johnson MR, Shinkareva SV, Millin R, Grossman M, Bhattacharyya R, Yacoub E.., | Extracting sentence word order and timing information from a natural reading experiment using multiband accelerated fMRI | OHBM | 2015 |
| Chelian, S. E., Paik, J., Pirolli, P., Lebiere, C., Bhattacharyya, R. | Reinforcement learning and instance-based learning approaches to modeling human decision making in a prognostic foraging task | Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob) (pp. 116-122) IEEE | August 2015 |
| M. Mansouri, V. De Sapio, and J. Reinbolt | Prioritized task-based control of movement with supporting contacts using OpenSim and Matlab | In Abstracts for the XV International Symposium on Computer Simulation in Biomechanics | July 2015 |
| Howard, M. D., Bhattacharyya, R., Chelian, S. E., Phillips, M. E., Pilly, P. K., Ziegler, M. D., Wang, H. | The neural basis of decision-making during sensemaking: implications for human-system interaction | IEEE Aerospace Conference (pp. 1-16) IEEE | March 2015 |
| V. De Sapio, N. Srinivasa | A methodology for controlling motion and constraint forces in holonomically constrained systems | Multibody System Dynamics, 33(2):179– 204 | February 2015 |
| V. De Sapio, D. Earl, R. Green, K. Saul | Human factors simulation using demographically tuned biomechanical models | In Proceedings of the 2014 International Annual Meeting of the Human Factors and Ergonomics Society, volume 58, pages 944– 948 | October 2014 |
| S. Goldfarb, D. Earl, V. De Sapio, M. Mansouri, J. Reinbolt | An approach and implementation for coupling neurocognitive and neuromechanical models | In Proceedings of the 2014 IEEE International Conference on Systems, Man, and Cybernetics, pages 406–413 | October 2014 |
| S. Goldfarb, R. Bhattacharyya, V. De Sapio | Coupled models of cognition and action: Behavioral phenotypes in the collective | In Poster Session for Collective Intelligence 2014, MIT | June 2014 |
| V. De Sapio | An approach for goal-oriented neuromuscular control of digital humans | International Journal of Human Factors Modelling and Simulation, 4(2):121– 144 | 2014 |
| O’Brien, M. J., Keegan, M. S., Goldstein, T., Millin, R., Benvenuto, J., Kay, K., Bhattacharyya, R. | Sparse Atomic Feature Learning via Gradient Regularization: With Applications to Finding Sparse Representations of fMRI Activity Patterns | IEEE Signal Processing in Medicine and Biology Symposium (SPMB) doi:10.1109/SPMB.2014.7002972 | 2014 |
| Chelian, S. E., Ziegler, M. D., Pirolli, P., Bhattacharyya, R. | Learning to prognostically forage in a neural network model of the interactions between neuromodulators and prefrontal cortex | Procedia Computer Science, 41, 32-39 | 2014 |
| Phillips, M., Ziegler, M. | A neurostimulation-based advanced training system for human performance augmentation | Brain Stimulation: Basic, Translational, and Clinical Research in Neuromodulation, 7(2), e11-e12 | 2014 |
| Pilly, P. K. and Grossberg, S. | How does the modular organization of entorhinal grid cells develop? | Frontiers in Human Neuroscience, 8, 337 | 2014 |
Click to show/hide the list of Papers (Under Review)
| Authors | Title | Publication | Year |
|---|---|---|---|
| PK Pilly, MD Howard, R Bhattacharyya | Contextual modulation of memory associations in the hippocampus | Frontiers in Human Neuroscience 12 | Nov 2018 |
| D Korchev, A Jammalamadaka, R Bhattacharyya | Automatic Rule Learning for Autonomous Driving Using Semantic Memory | arXiv preprint arXiv:1809.07904 | Sep 2018 |
| Jones AP, Choe J, Bryant NB, Robinson CSH, Ketz NA, Skorheim SW, Combs A, Lamphere ML, Robert B, Gill HA Heinrich MD, Howard MD, Clark VP, Pilly PK | Closed-Loop tACS Delivered During Slow-Wave Sleep Enhances Consolidation of Generalized Information | Submitted to Journal of Cognitive Neuroscience | 2018 |
| D Khosla, D Huber, R Bhattacharyya | Optimized static and video EEG rapid serial visual presentation (RSVP) paradigm based on motion | SPIE Defense + Security | May 2017 |
| D Khosla, R Bhattacharyya | Surprise computation | Signal Processing, Sensor/Information Fusion, and Target Recognition XXVI | 2017 |
| H Moon, J Benvenuto, R Bhattacharyya | Nonlinear semantic space based on lexical graph | Future Technologies Conference (FTC), 99-102 | 2016 |
Click to show/hide the list of Patents
| Authors | Title | Patent # | Date |
|---|---|---|---|
| V. De Sapio and D. .J. Earl | Method and system for tuning a musculoskeletal model | US 9,858,391 | January 2, 2018 |
| H. Hoffmann, D. W. Payton, V. De Sapio | Method for tele-robotic operations over time-delayed communication links | US 9,776,325 | October 3, 2017 |
| V. De Sapio, M. Howard, R. Green | Quantifying muscle and tendon fatigue during physical exertion | US 9,610,036 | April 4, 2017 |
| V. De Sapio, M. D. Howard, R. F. Green | Quantifying muscle and tendon fatigue during physical exertion | US 9,610,036 | March 4, 2017 |
| V. De Sapio, N. Srinivasa | System for controlling brain machine interfaces and neural prosthetic systems | US 9,566,174 | February 14, 2017 |
| M. Howard | System and Method to discover and encode indirect associations in associative memory | US 9,558,825 | January 31, 2017 |
| V. De Sapio, H. Hoffmann | System for controlling motion and constraint forces in a robotic system | US 9,364,951 | June 14, 2016 |
| M. Daily, M. Howard, Y. Chen, D. Payton, R. Sundareswara | Recall system using spiking neuron networks | US 9,020,870 | April 28, 2015 |
| M. Howard, R. Bhattacharyya | System and Method for Adaptive Recall | US 9,002,762 | April 7, 2015 |
| M. Howard, R. Bhattacharyya | Framework for flexible cognitive perception and action selection | US 8,990,139 | March 24, 2015 |
| M. Daily, M. Howard, Y. Chen, R. Sundareswara, D. Payton | System for representing, storing, and reconstructing an input signal | US 8,756,183 | June 17, 2014 |
News
- Pilly, P. K. (2018). Strengthening individual episodic memories in virtual reality using transcranial current stimulation in sleep. 15th Annual World Congress of the Society for Brain Mapping and Therapeutics, Los Angeles, CA. (Invited Talk in the session on “Augmented and Virtual Reality Clinical Neuroscience”).
- Pilly, P. K. (2018). Waking tagging and sleep cueing of episodic memories in virtual reality using non-invasive electrical stimulation. 25th Joint Symposium on Neural Computation, California Institute of Technology, Pasadena, CA. (Invited Talk).
- Ketz et al. (Journal of Neuroscience, 2018) article was featured in Digital Trends, Reuters Health, Popular Science, Medical News Today, The Neurocritic, The Quebec Post, Forbes, Live Science, KPCC 89.3, Europa Press, Medscape, Society for Neuroscience, Digital Journal, among others.
- Ketz et al. (Journal of Neuroscience, 2018) article has an Altmetric Attention Score of 492 (#14 of 17,630 articles published by the Journal of Neuroscience so far; 99th percentile for High Attention Score; Top 5% of all research outputs scored by Altmetric).
- Krause et al. (Current Biology, 2017) article was featured in Newsweek, Big Think, Daily Mail, Next Big Future, Yahoo! News, Futurism, Electronic Products, The Daily New Nation, The Scientist, among others.
- Krause et al. (Current Biology, 2017) article was recommended in F1000 Prime as being of special significance in its field by F1000 Faculty Member Brian Corneil, and was rated as “New Finding, Technical Advance”.
- Pilly and Grossberg (Frontiers in Human Neuroscience, 2014) was among the Top 10 most viewed Psychology research articles from Frontiers journals in June 2014 (Rank: 8).
Press Releases
Events
Upcoming Events
 |
Workshop on Human Machine SystemsAugust 23-24, 2018 | HRL Laboratories - Malibu, CA |
Past Events
October 11, 2017 | HRL Laboratories - Malibu, CA
Workshop on Memory Consolidation, Restoration, and Augmentation
Current Openings for CHMC
Contact
Email: pkpilly[at]hrl.com
Dr. Praveen Pilly
Senior Research Staff Scientist
Leader, Center for Human Machine Collaboration
HRL Laboratories, LLC
3011 Malibu Canyon Road
Malibu, CA 90265
USA
