Browsing by Author "Karpatne, Anuj"

Now showing 1 - 20 of 63
  • Thumbnail Image
    Achieving More with Less: Learning Generalizable Neural Networks With Less Labeled Data and Computational Overheads
    Bu, Jie (Virginia Tech, 2023-03-15)
    Recent advancements in deep learning have demonstrated its incredible ability to learn generalizable patterns and relationships automatically from data in a number of mainstream applications. However, the generalization power of deep learning methods largely comes at the costs of working with very large datasets and using highly compute-intensive models. Many applications cannot afford these costs needed to ensure generalizability of deep learning models. For instance, obtaining labeled data can be costly in scientific applications, and using large models may not be feasible in resource-constrained environments involving portable devices. This dissertation aims to improve efficiency in machine learning by exploring different ways to learn generalizable neural networks that require less labeled data and computational resources. We demonstrate that using physics supervision in scientific problems can reduce the need for labeled data, thereby improving data efficiency without compromising model generalizability. Additionally, we investigate the potential of transfer learning powered by transformers in scientific applications as a promising direction for further improving data efficiency. On the computational efficiency side, we present two efforts for increasing parameter efficiency of neural networks through novel architectures and structured network pruning.
  • Thumbnail Image
    ACM Venue Recommender System
    Kodur Kumar, Harinni (Virginia Tech, 2020-06-17)
    A frequent goal of a researcher is to publish his/her work in appropriate conferences and journals. With a large number of options for venues in the microdomains of every research discipline, the issue of selecting suitable locations for publishing cannot be underestimated. Further, the venues diversify themselves in the form of workshops, symposiums, and challenges. Several publishers such as IEEE and Springer have recognized the need to address this issue and have developed journal recommenders. In this thesis, our goal is to design and develop a similar recommendation system for the ACM dataset. We view this recommendation problem from a classification perspective. With the success of deep learning classifiers in recent times and their pervasiveness in several domains, we modeled several 1D Convolutional neural network classifiers for the different venues. When given some submission information like title, keywords, abstract, etc. about a paper, the recommender uses these developed classifier predictions to recommend suitable venues to the user. The dataset used for the project is the ACM Digital Library metadata that includes textual information for research papers and journals submitted at various conferences and journals over the past 60 years. We developed the recommender based on two approaches: 1) A binary CNN classifier per venue (single classifiers), and 2) Group CNN classifiers for venue groups (group classifiers). Our system has achieved a MAP of 0.55 and 0.51 for single and group classifiers. We also show that our system has a high recall rate.
  • Thumbnail Image
    Adjoint-Matching Neural Network Surrogates for Fast 4D-Var Data Assimilation
    Chennault, Austin; Popov, Andrey A.; Subrahmanya, Amit N.; Cooper, Rachel; Karpatne, Anuj; Sandu, Adrian (2021-11-16)
    The data assimilation procedures used in many operational numerical weather forecasting systems are based around variants of the 4D-Var algorithm. The cost of solving the 4D-Var problem is dominated by the cost of forward and adjoint evaluations of the physical model. This motivates their substitution by fast, approximate surrogate models. Neural networks offer a promising approach for the data-driven creation of surrogate models. The accuracy of the surrogate 4D-Var problem’s solution has been shown to depend explicitly on accurate modeling of the forward and adjoint for other surrogate modeling approaches and in the general nonlinear setting. We formulate and analyze several approaches to incorporating derivative information into the construction of neural network surrogates. The resulting networks are tested on out of training set data and in a sequential data assimilation setting on the Lorenz-63 system. Two methods demonstrate superior performance when compared with a surrogate network trained without adjoint information, showing the benefit of incorporating adjoint information into the training process.
  • Thumbnail Image
    Adversarial Learning based framework for Anomaly Detection in the context of Unmanned Aerial Systems
    Bhaskar, Sandhya (Virginia Tech, 2020-06-18)
    Anomaly detection aims to identify the data samples that do not conform to a known normal (regular) behavior. As the definition of an anomaly is often ambiguous, unsupervised and semi-supervised deep learning (DL) algorithms that primarily use unlabeled datasets to model normal (regular) behaviors, are popularly studied in this context. The unmanned aerial system (UAS) can use contextual anomaly detection algorithms to identify interesting objects of concern in applications like search and rescue, disaster management, public security etc. This thesis presents a novel multi-stage framework that supports detection of frames with unknown anomalies, localization of anomalies in the detected frames, and validation of detected frames for incremental semi-supervised learning, with the help of a human operator. The proposed architecture is tested on two new datasets collected for a UAV-based system. In order to detect and localize anomalies, it is important to both model the normal data distribution accurately as well as formulate powerful discriminant (anomaly scoring) techniques. We implement a generative adversarial network (GAN)-based anomaly detection architecture to study the effect of loss terms and regularization on the modeling of normal (regular) data and arrive at the most effective anomaly scoring method for the given application. Following this, we use incremental semi-supervised learning techniques that utilize a small set of labeled data (obtained through validation from a human operator), with large unlabeled datasets to improve the knowledge-base of the anomaly detection system.
  • Thumbnail Image
    aiWATERS: An Artificial Intelligence Framework for the Water Sector
    Vekaria, Darshan (Virginia Tech, 2023-07-20)
    The ubiquity of Artificial Intelligence (AI) and Machine Learning (ML) applications has led to their widespread adoption across diverse domains like education, self-driving cars, healthcare, and more. AI is making its way into the industry, beyond research and academia. Concurrently, the water sector is undergoing a digital transformation, driven by challenges such as water demand forecasting, wastewater treatment, asset maintenance and management, and water quality assessment. Water utilities are at different stages in their journey of digital transformation, and its decision-makers, who are non-expert stakeholders in AI applications, must understand the technology to make informed decisions. The non-expert stakeholders should know that while AI has numerous benefits to offer, there are also many challenges related to data, model development, knowledge integration, and ethical concerns that should be considered before implementing it for real-world applications. Civil engineering is a licensed profession where critical decision-making is involved. Failure of critical decisions by civil engineers may put their license at risk, and therefore trust in any decision-support technology is crucial for its acceptance in real-world applications. This research proposes a framework called aiWATERS (Artificial Intelligence for the Water Sector) to facilitate the successful application of AI in the water sector. Based on this framework, we conduct pilot interviews and surveys with various small, medium, and large water utilities to capture their current state of AI implementation and identify the challenges faced by them. The research findings reveal that most of the water utilities are at an early stage of implementing AI as they face concerns regarding the blackbox nature, trustworthiness, and sustainability of AI technology in their system. The aiWATERS framework is intended to help the utilities navigate through these issues in their journey of digital transformation.
  • Thumbnail Image
    Assessing Structure–Property Relationships of Crystal Materials using Deep Learning
    Li, Zheng (Virginia Tech, 2020-08-05)
    In recent years, deep learning technologies have received huge attention and interest in the field of high-performance material design. This is primarily because deep learning algorithms in nature have huge advantages over the conventional machine learning models in processing massive amounts of unstructured data with high performance. Besides, deep learning models are capable of recognizing the hidden patterns among unstructured data in an automatic fashion without relying on excessive human domain knowledge. Nevertheless, constructing a robust deep learning model for assessing materials' structure-property relationships remains a non-trivial task due to highly flexible model architecture and the challenge of selecting appropriate material representation methods. In this regard, we develop advanced deep-learning models and implement them for predicting the quantum-chemical calculated properties (i.e., formation energy) for an enormous number of crystal systems. Chapter 1 briefly introduces some fundamental theory of deep learning models (i.e., CNN, GNN) and advanced analysis methods (i.e., saliency map). In Chapter 2, the convolutional neural network (CNN) model is established to find the correlation between the physically intuitive partial electronic density of state (PDOS) and the formation energies of crystals. Importantly, advanced machine learning analysis methods (i.e., salience mapping analysis) are utilized to shed light on underlying physical factors governing the energy properties. In Chapter 3, we introduce the methodology of implementing the cutting-edge graph neural networks (GNN) models for learning an enormous number of crystal structures for the desired properties.
  • Thumbnail Image
    Augmented Neural Network Surrogate Models for Polynomial Chaos Expansions and Reduced Order Modeling
    Cooper, Rachel Gray (Virginia Tech, 2021-05-20)
    Mathematical models describing real world processes are becoming increasingly complex to better match the dynamics of the true system. While this is a positive step towards more complete knowledge of our world, numerical evaluations of these models become increasingly computationally inefficient, requiring increased resources or time to evaluate. This has led to the need for simplified surrogates to these complex mathematical models. A growing surrogate modeling solution is with the usage of neural networks. Neural networks (NN) are known to generalize an approximation across a diverse dataset and minimize the solution along complex nonlinear boundaries. Additionally, these surrogate models can be found using only incomplete knowledge of the true dynamics. However, NN surrogates often suffer from a lack of interpretability, where the decisions made in the training process are not fully understood, and the roles of individual neurons are not well defined. We present two solutions towards this lack of interpretability. The first focuses on mimicking polynomial chaos (PC) modeling techniques, modifying the structure of a NN to produce polynomial approximations of the underlying dynamics. This methodology allows for an extractable meaning from the network and results in improvement in accuracy over traditional PC methods. Secondly, we examine the construction of a reduced order modeling scheme using NN autoencoders, guiding the decisions of the training process to better match the real dynamics. This guiding process is performed via a physics-informed (PI) penalty, resulting in a speed-up in training convergence, but still results in poor performance compared to traditional schemes.
  • Thumbnail Image
    A Bayesian Framework for Multi-Stage Robot, Map and Target Localization
    Papakis, Ioannis (Virginia Tech, 2019)
    This thesis presents a generalized Bayesian framework for a mobile robot to localize itself and a target, while building a map of the environment. The proposed technique builds upon the Bayesian Simultaneous Robot Localization and Mapping (SLAM) method, to allow the robot to localize itself and the environment using map features or landmarks in close proximity. The target feature is distinguished from the rest of features since the robot has to navigate to its location and thus needs to be observed from a long distance. The contribution of the proposed approach is on enabling the robot to track a target object or region, using a multi-stage technique. In the first stage, the target state is corrected sequentially to the robot correction in the Recursive Bayesian Estimation. In the second stage, with the target being closer, the target state is corrected simultaneously with the robot and the landmarks. The process allows the robot's state uncertainty to be propagated into the estimated target's state, bridging the gap between tracking only methods where the target is estimated assuming known observer state and SLAM methods where only landmarks are considered. When the robot is located far, the sequential stage is efficient in tracking the target position while maintaining an accurate robot state using close only features. Also, target belief is always maintained in comparison to temporary tracking methods such as image-tracking. When the robot is closer to the target and most of its field of view is covered by the target, it is shown that simultaneous correction needs to be used in order to minimize robot, target and map entropies in the absence of other landmarks.
  • Thumbnail Image
    Benchmarking Methods For Predicting Phenotype Gene Associations
    Tyagi, Tanya (Virginia Tech, 2020-09-16)
    Assigning human genes to diseases and related phenotypes is an important topic in modern genomics. Human Phenotype Ontology (HPO) is a standardized vocabulary of phenotypic abnormalities that occur in human diseases. Computational methods such as label-propagation and supervised-learning address challenges posed by traditional approaches such as manual curation to link genes to phenotypes in the HPO. It is only in recent years that computational methods have been applied in a network-based approach for predicting genes to disease-related phenotypes. In this thesis, we present an extensive benchmarking of various computational methods for the task of network-based gene classification. These methods are evaluated on multiple protein interaction networks and feature representations. We empirically evaluate the performance of multiple prediction tasks using two evaluation experiments: cross-fold validation and the more stringent temporal holdout. We demonstrate that all of the prediction methods considered in our benchmarking analysis have similar performance, with each of the methods outperforming a random predictor.
  • Thumbnail Image
    Beyond Discriminative Regions: Saliency Maps as Alternatives to CAMs for Weakly Supervised Semantic Segmentation
    Maruf, M.; Daw, Arka; Dutta, Amartya; Bu, Jie; Karpatne, Anuj (2023)
  • Thumbnail Image
    Combining Data-driven and Theory-guided Models in Ensemble Data Assimilation
    Popov, Andrey Anatoliyevich (Virginia Tech, 2022-08-23)
    There once was a dream that data-driven models would replace their theory-guided counterparts. We have awoken from this dream. We now know that data cannot replace theory. Data-driven models still have their advantages, mainly in computational efficiency but also providing us with some special sauce that is unreachable by our current theories. This dissertation aims to provide a way in which both the accuracy of theory-guided models, and the computational efficiency of data-driven models can be combined. This combination of theory-guided and data-driven allows us to combine ideas from a much broader set of disciplines, and can help pave the way for robust and fast methods.
  • Thumbnail Image
    Comparative Analysis of Genomic Similarity Tools in Species Identification
    Nerella, Chandra Sekhar (Virginia Tech, 2025-01-14)
    This study presents the development and evaluation of an automated pipeline for genome comparison, leveraging four bioinformatics tools: alignment-based methods (pyANI, Fas- tANI) and k-mer-based methods (Sourmash, BinDash 2.0). The analysis focuses on high- quality genomic datasets characterized by 100% completeness, ensuring consistency and accuracy in the comparison process. The pipeline processes genomes under uniform con- ditions, recording key performance metrics such as execution time and rank correlations. Initial comparisons were conducted on a subset of five genomes, generating 10 unique pair- wise comparisons to establish baseline performance. This preliminary analysis identified k = 10 as the optimal k-mer size for Sourmash and BinDash, significantly improving their comparability with alignment-based methods. For the expanded dataset of 175 genomes, encompassing (175C2) = 15,225 unique comparisons, pyANI and FastANI demonstrated high similarity values, often exceeding 90% for closely related genomes. Rank correlations, calculated using Spearman's ρ and Kendall's τ , high- lighted strong agreement between pyANI and FastANI (ρ = 0.9630 , τ = 0.8625) due to their shared alignment-based methodology. Similarly, Sourmash and BinDash, both employing k-mer-based approaches, exhibited moderate-to-strong rank correlations (ρ = 0.6967, τ = 0.5290). In contrast, the rank correlations between alignment-based and k-mer-based tools were lower, underscoring methodological differences in genome similarity calculations. Execution times revealed significant contrasts between the tools. Alignment-based meth- ods required substantial computation time, with pyANI taking an average of 1.97 seconds per comparison and FastANI averaging 0.81 seconds per comparison. Conversely, k-mer- based methods demonstrated exceptional computational efficiency, with Sourmash complet- ing comparisons in 2.1 milliseconds and BinDash in just 0.25 milliseconds per comparison, reflecting a difference of nearly three orders of magnitude between the two categories. These results underscore the trade-offs between computational cost and methodological approaches in genome similarity estimation. This study provides valuable insights into the relative strengths and weaknesses of genome comparison tools, offering a comprehensive framework for selecting appropriate methods for diverse genomic research applications. The findings emphasize the importance of param- eter optimization for k-mer-based tools and highlight the scalability of these methods for large-scale genomic analyses.
  • Thumbnail Image
    Data Sharing and Retrieval of Manufacturing Processes
    Seth, Avi (Virginia Tech, 2023-03-28)
    With Industrial Internet, businesses can pool their resources to acquire large amounts of data that can then be used in machine learning tasks. Despite the potential to speed up training and deployment and improve decision-making through data-sharing, rising privacy concerns are slowing the spread of such technologies. As businesses are naturally protective of their data, this poses a barrier to interoperability. While previous research has focused on privacy-preserving methods, existing works typically consider data that is averaged or randomly sampled by all contributors rather than selecting data that are best suited for a specific downstream learning task. In response to the dearth of efficient data-sharing methods for diverse machine learning tasks in the Industrial Internet, this work presents an end-to end working demonstration of a search engine prototype built on PriED, a task-driven data-sharing approach that enhances the performance of supervised learning by judiciously fusing shared and local participant data.
  • Thumbnail Image
    A Deep Learning Approach to Predict Full-Field Stress Distribution in Composite Materials
    Sepasdar, Reza (Virginia Tech, 2021-05-17)
    This thesis proposes a deep learning approach to predict stress at various stages of mechanical loading in 2-D representations of fiber-reinforced composites. More specifically, the full-field stress distribution at elastic and at an early stage of damage initiation is predicted based on the microstructural geometry. The required data set for the purposes of training and validation are generated via high-fidelity simulations of several randomly generated microstructural representations with complex geometries. Two deep learning approaches are employed and their performances are compared: fully convolutional generator and Pix2Pix translation. It is shown that both the utilized approaches can well predict the stress distributions at the designated loading stages with high accuracy.
  • Thumbnail Image
    Deep Learning for Forest Plantation Mapping in Godavari Districts of Andhra Pradesh, India
    More, Snehal; Karpatne, Anuj; Wynne, Randolph H.; Thomas, Valerie A. (Virginia Tech, 2019-08)
    Small-area forest plantations play a vital role in the socioeconomic well-being of farmers in Southeast Asia. Most plantations are established on former agricultural land, often on land less suitable for agriculture. Plantations that are converted from natural forest have adverse impacts on biodiversity. Mapping small-area plantations is thus important to understand the dynamics of forest cover in Southeast Asia and to study the social, economic, and ecological effects of this important land cover and land use change. While the small size of forest plantations makes it difficult to detect them using moderate resolution satellite sensors, the problem is exacerbated by the high degree of mixing between plantations, surrounding vegetation, and other land covers, which often show variegated responses in satellite signals across space and time. In this paper, we study the problem of mapping small-area forest plantations in East and West Godavari districts of Andhra Pradesh, India using deep learning methods. Remotely sensed cloud-free data from the Harmonized Landsat Sentinel-2 S10 product were classified using a pixel-level neural network and training data labeled using a field-based survey in concert with expert aerial photo interpretation. We compare the performance of deep learning methods with a baseline random forest classifier in our study region of 21543 sq. km over a period of 3 years and analyze the differences in the results across land cover classes and seasons.
  • Thumbnail Image
    Deep Multi-Resolution Operator Networks (DMON): Exploring Novel Data-Driven Strategies for Chaotic Inverse Problems
    Donald, Sam Alexander Knowles (Virginia Tech, 2024-01-11)
    Inverse problems, foundational in applied sciences, involve deducing system inputs from specific output observations. These problems find applications in diverse domains such as aerospace engineering, weather prediction, and oceanography. However, their solution often requires complex numerical simulations and substantial computational resources. Modern machine learning based approaches have emerged as an alternative and flexible methodology for solving these types of problems, however their generalization power often comes at the cost of working with large descriptive datasets, a requirement that many applications cannot afford. This thesis proposes and explores the novel Deep Multi-resolution Operator Network (DMON), inspired by the recently developed DeepONet architecture. The DMON model is designed to solve inverse problems related to chaotic non-linear systems with low-resolution data through intelligently utilizing high-resolution data from a similar system. Performance of the DMON model and the proposed selection mechanisms are evaluated on two chaotic systems, a double pendulum and turbulent flow around a cylinder, with improvements observed under idealized scenarios whereby high and low-resolution inputs are manually paired, along with minor improvements when this pairing is conducted through the proposed the latent space comparison selection mechanism.
  • Thumbnail Image
    DeepARG+ - A Computational Pipeline for the Prediction of Antibiotic Resistance
    Kulkarni, Rutwik Shashank (Virginia Tech, 2021-06-16)
    The global spread of antibiotic resistance warrants concerted surveillance in the clinic and in the environment. The widespread use of metagenomics for various studies has led to the generation of a large amount of sequencing data. Next-generation sequencing of microbial communities provides an opportunity for proactive detection of emerging antibiotic resistance genes (ARGs) from such data, but there are a limited number of pipelines that enable the identification of novel ARGs belonging to diverse antibiotic classes at present. Therefore, there is a need for the development of computational pipelines that can identify these putative novel ARGs. Such pipelines should be scalable, accessible and have good performance. To address this problem we develop a new method for predicting novel ARGs from genomic or metagenomic sequences, leveraging known ARGs of different resistance categories. Our method takes into account the physio-chemical properties that are intrinsic to different ARG families. Traditionally, new ARGs are predicted by making sequence alignment and calculating sequence similarity to existing ARG reference databases, which can be very time consuming. Here we introduce an alignment free and deep learning prediction method that incorporates both the primary protein sequences of ARGs and their physio-chemical properties. We compare our method with existing pipelines including hidden Markov model based Resfams and fARGene, sequence alignment and machine learning-based DeepARG-LS, and homology modelling based Pairwise Comparative Modelling. We also use our model to detect novel ARGs from various environments including human-gut, soil, activated sludge and the influent samples collected from a waste water treatment plant. Results show that our method achieves greater accuracy compared to existing models for the prediction of ARGs and enables the detection of putative novel ARGs, providing promising targets for experimental characterization to the scientific community.
  • Thumbnail Image
    Deidentification of Face Videos in Naturalistic Driving Scenarios
    Thapa, Surendrabikram (Virginia Tech, 2023-09-05)
    The sharing of data has become integral to advancing scientific research, but it introduces challenges related to safeguarding personally identifiable information (PII). This thesis addresses the specific problem of sharing drivers' face videos for transportation research while ensuring privacy protection. To tackle this issue, we leverage recent advancements in generative adversarial networks (GANs) and demonstrate their effectiveness in deidentifying individuals by swapping their faces with those of others. Extensive experimentation is conducted using a large-scale dataset from ORNL, enabling the quantification of errors associated with head movements, mouth movements, eye movements, and other human factors cues. Additionally, qualitative analysis using metrics such as PERCLOS (Percentage of Eye Closure) and human evaluators provide valuable insights into the quality and fidelity of the deidentified videos. To enhance privacy preservation, we propose the utilization of synthetic faces as substitutes for real faces. Moreover, we introduce practical guidelines, including the establishment of thresholds and spot checking, to incorporate human-in-the-loop validation, thereby improving the accuracy and reliability of the deidentification process. In addition to this, this thesis also presents mitigation strategies to effectively handle reidentification risks. By considering the potential exploitation of soft biometric identifiers or non-biometric cues, we highlight the importance of implementing comprehensive measures such as robust data user licenses and privacy protection protocols.
  • Thumbnail Image
    Developing a Computational Pipeline for Detecting Multi-Functional Antibiotic Resistance Genes in Metagenomics Data
    Dang, Ngoc Khoi (Virginia Tech, 2022-06-09)
    Antibiotic resistance is currently a global threat spanning clinical, environmental, and geopolitical research domains. The environment is increasingly recognized as a key node in the spread of antibiotic resistance genes (ARGs), which confer antibiotic resistance to bacteria. Detecting ARGs in the environment is the first step in monitoring and controlling antibiotic resistance. In recent years, next-generation sequencing of environmental samples (metagenomic sequencing data) has become a prolific tool for the field of surveillance. Metagenomic data are nucleic acid sequences, or nucleotides, of environmental samples. Metagenomic sequencing data has been used over the years to detect and analyze ARGs. An intriguing instance of ARGs is the multi-functional ARG, where one ARG encodes two or more different antibiotic resistance functions. Multi-functional ARGs provide resistance to two or more antibiotics, thus should have evolutionary advantage over ARGs with resistance to single antibiotic. However, there is no tool readily available to detect these multi-functional ARGs in metagenomic data. In this study, we develop a computational pipeline to detect multi-functional ARGs in metagenomic data. The pipeline takes raw metagenomic data as the input and generates a list of potential multi-functional ARGs. A plot for each potential multi-functional ARG is also created, showing the location of the multi-functionalities in the sequence and the sequencing coverage level. We collected samples from three different sources: influent samples of a wastewater treatment plant, hospital wastewater samples, and reclaimed water samples, ran the pipeline, and identified 19, 57, and 8 potentially bi-functional ARGs in each source, respectively. Manual inspection of the results identified three most likely bi-functional ARGs. Interestingly, one bi-functional ARG, encoding both aminoglycoside and tetracycline resistance, appeared in all three data sets, indicating its prevalence in different environments. As the amount of antibiotics keeps increasing in the environment, multi-functional ARGs might become more and more common. The pipeline will be a useful computational tool for initial screening and identification of multi-functional ARGs in metagenomic data.
  • Thumbnail Image
    Discovering Novel Biological Traits From Images Using Phylogeny-Guided Neural Networks
    Elhamod, Mohannad; Khurana, Mridul; Manogaran, Harish Babu; Uyeda, Josef C.; Balk, Meghan A.; Dahdul, Wasila; Bakış, Yasin; Bart, Henry L. Jr.; Mabee, Paula M.; Lapp, Hilmar; Balhoff, James P.; Charpentier, Caleb; Carlyn, David; Chao, Wei-Lun; Stewart, Charles V.; Rubenstein, Daniel I.; Berger-Wolf, Tanya; Karpatne, Anuj (ACM, 2023-08-06)
    Discovering evolutionary traits that are heritable across species on the tree of life (also referred to as a phylogenetic tree) is of great interest to biologists to understand how organisms diversify and evolve. However, the measurement of traits is often a subjective and labor-intensive process, making trait discovery a highly label-scarce problem. We present a novel approach for discovering evolutionary traits directly from images without relying on trait labels. Our proposed approach, Phylo-NN, encodes the image of an organism into a sequence of quantized feature vectors–or codes–where different segments of the sequence capture evolutionary signals at varying ancestry levels in the phylogeny. We demonstrate the effectiveness of our approach in producing biologically meaningful results in a number of downstream tasks including species image generation and species-to-species image translation, using fish species as a target example.