File size: 1,804 Bytes
ca8858c efb420c ca8858c efb420c f1a9699 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 |
---
license: cc-by-nc-sa-4.0
language:
- en
pipeline_tag: image-feature-extraction
tags:
- pathology
- neuropathology
- foundation_model
- vit
---
# neuroFM_HE20x
ViT-large (300M parameters) trained on a diverse neuropathology dataset.
## Model Usage
To get started, first clone the repository with this command:
```bash
git clone --no-checkout https://huggingface.co/MountSinaiCompPath/neuroFM_HE20x && cd neuroFM_HE20x && git sparse-checkout init --no-cone && git sparse-checkout set '/*' '!*.bin' && git checkout
```
Now you can use the following code:
```python
from PIL import Image
import numpy as np
import vision_transformer
import torch
import torch.nn as nn
import torchvision.transforms as transforms
from huggingface_hub import PyTorchModelHubMixin
class neuroFM_HE20x(nn.Module, PyTorchModelHubMixin):
def __init__(self):
super().__init__()
vit_kwargs = dict(
img_size=224,
patch_size=14,
init_values=1.0e-05,
ffn_layer='swiglufused',
block_chunks=4,
qkv_bias=True,
proj_bias=True,
ffn_bias=True,
)
self.encoder = vision_transformer.__dict__['vit_large'](**vit_kwargs)
def forward(self, x):
return self.encoder(x)
# Download model
model = neuroFM_HE20x.from_pretrained("MountSinaiCompPath/neuroFM_HE20x")
# Set up transform
transform = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
])
# Image
img = np.random.randint(0, 256, size=224*224*3).reshape(224,224,3).astype(np.uint8)
img = Image.fromarray(img)
img = transform(img).unsqueeze(0)
# Inference
with torch.no_grad():
h = model(img)
``` |