Umar Masud

Hi 👋! I am a researcher and developer in the field of AI/ML.
I am a recent graduate (MSc) from the University of Toronto 🍁 and a past intern at Samsung AI Center Toronto, working with the camera technology team. Prior to that, I did my undergrad in Computer Engineering at JMI University, New Delhi.

I have a strong background in AI, with a focus on multimedia data. My work includes top-conference paper submissions (CVPR 25' under review), experience with large-scale data (millions), prototyping ML models and understanding the complete ML lifecycle. I have done many personal projects and academic internships in the ML domain, solving a variety of problems. Currently exploring vision+language solutions like Visual-RAG, Multimodal VLMs, LLMs, Agents, Diffusion, etc. 🚀

My interest is in making machines understand vision & language as we humans do.
I love to make and own end-to-end projects that solve real-life problems.

Additionally, I am enthusiastic about innovation, ideas, entrepreneurship, community, and the like. A keen interest in the start-up ecosystem.

Education

Applied Research

PUBLICATIONS

1. Self-Supervised Learning Masud, U., Cohen, E., Bendidi, I., Bollot, G., Genovesio, A. (2022). Comparison of semi-supervised learning methods for High Content Screening quality control. BioImage Computing workshop at ECCV 2022. https://doi.org/10.48550/arXiv.2208.04592
2. Synthetic Data Generation Efficient ML Masud, U., Siddiqui, M., Sadiq, Mohd., Masood, S. (2022). SCS-Net: An efficient and practical approach towards Face Mask Detection. Procedia Computer Science Journal. ICMLDE, 2022. https://doi.org/10.1016/j.procs.2023.01.165
3. Domain Generalisation Contrastive Learning Jambigi, C., Masud, U., Chakraborty, A. (2022). G-PReDICT: Generalizable Person Re-ID using Domain Invariant Contrastive Techniques. ICVGIP, 2022. https://doi.org/10.1145/3571600.3571655
4. Inverse Problem Masud, U., Shwenker, F. (2022). Compressed Image Super-Resolution using Pre-trained Model Assistance. COMSYS, 2022. https://doi.org/10.1007/978-981-99-2680-0_5
5. Video classification Masud, U., Sadiq, Mohd., Masood, S., Ahmad, M., and Ahmed A. Abd El-Latif. 2023. LW-DeepFakeNet: A Lightweight Time Distributed CNN-LSTM network for real-time DeepFake Video Detection.Signal, Image and Video Processing. https://doi.org/10.1007/s11760-023-02633-9
6. Media Authenticity Masud, U., Agarwal, A. (2021). Analysing Statistical methods for Automatic Detection of Image Forgery. arXiv. https://doi.org/10.48550/arXiv.2111.12661

Skills

• Classical ML/DL,
• Computer Vision,
• Multi-Modal learning,
• Generative AI,
• VLMs,
• LLMs,
• Agents,
• Diffusion.

• Python,
• Java,
• Bash/Shell.

• NumPy,
• Pandas,
• Matplotlib,
• Sklearn,
• OpenCV,
• Keras,
• TensorFlow,
• PyTorch,
• Flask.

• MySql,
• ChromaDB (vector)

• Git,
• MLflow,
• W&B,
• Tensorboard,

Projects

This is a collective dump of all my projects, big and small. Some of them were part of coursework, others for self learning. The advanced ones are marked by 🔥.

• Generative AI
  • 🔥 Built a Multi-Modal Visual-RAG for intelligent gallery search that combines image and metadata search capabilities using multimodal embeddings and evaluation.
• Classical ML
  • 🔥 Reproduced 5 papers on the topic Image Forgery Detection that uses handcrafted features for classification of pristine and tampered images.
  • Diabetic Retinopathy Detection using Texture Features and Ensemble Learning (paper implementation). Achieved F1-score = 0.97 and accuracy = 97.2%.
  • Fog detection in images using GLCM based features and SVM (paper implementation). Got F1-score = 0.83 and test accuracy = 82.3%
  • Phishing URL detection system based on URL features using SVM (paper implementation). Achieved F1-score = 0.99 and test accuracy = 99.2%
• OpenCV Projects
  • 🔥 Air-Piano, an air-based piano enabling the person to play through hand(fingertip) movements.
  • Air-Drum System, an air-based drum beat generator.
  • Background Color Detection, uses 2 techniques to detect a suitable background for the input image.
• Deep Learning
  • 🔥 Clicking better Images with Under Display Cameras (UDC) in Smartphones. A 7.78M params model with KD gets 30.59 PSNR, and diffusion beats SOTA getting 42.37 PSNR. (Report)
  • Integrating ML functionalities - generating tags and descriptions for uploaded images, in an existing Instagram clone web-app in Flask. (Report)
  • 🔥 Different Descriptors for Squeeze and Excitation Attention Block - experimented with standard deviation, trace, largest singular value, and DC coefficient of DCT instead of usual GlobalAvgPool2d. The SVD approach gives a 0.78% improvement but with an 80% increase in training time. (Report)
  • Image Inpainting using a U-Net model with a fused ConvMixer encoder. The feature fusion method showed 1.34% improvement in terms of dice coefficient (Report)
  • Background Remover tool for portrait images of humans, made using a U-Net model trained for semantic segmentation of the image. The model achieved 0.981 IOU-score on test data. Also deployed on a web-app.
  • Implemented the paper - Medical image denoising using Convolutional Denoising Autoencoders(CAE). Achieved a loss = 0.106 or Structural Similarity Index(SSIM) = 0.894 .
  • Image similarity measure through Siamese network on fashion apparels. Got an evaluation accuracy of 94.2%
  • Plant Pathology Challenge, a FGVC8 workshop challenge at CVPR-2021 for multi-label classification of plant leaf diseases. Got 87.34 accuracy with a pre-trained model as feature extractor.
  • Human Emotion Detection, Pneumonia Prediction models.

Web Development

• Banking System
  • The project contains a simple banking system that enables to transact between the customers. It uses HTML, CSS, bootstrap, PHP, and MySQL, with the local server provided by the XAMPP.
• Website Template for InnerveSOC
  • As a part of the InnerveSOC competition, designed a complete website template for Innerve Tech-Fest 2020, IGDTUW. I was the adjudged winner.