LeViT pretraining of larger models still a WIP, they don't train well / easily without distillation.Currently testing for imminent PyPi 0.6.x release.previous impl exists as LayerNormExp2d in models/layers/norm.py.a bit slower than previous custom impl on some hardware (ie Ampere w/ CL), but overall fewer regressions across wider HW / PyTorch version ranges. ).permute(0, 3, 1, 2) via LayerNorm2d in all cases. Default ConvNeXt LayerNorm impl to use F.layer_norm(x.permute(0, 2, 3, 1).Add support to change image extensions scanned by timm datasets/parsers.Pretrained weights / configs can be loaded externally (ie from local disk) w/ support for head adaptation.Hugging Face Hub support fixes verified, demo notebook TBA.cs3, darknet, and vit_*relpos weights above all trained on TPU thanks to TRC program! Rest trained on overheating GPUs.My own model weight results (all ImageNet-1k training).Better than original small, but not their new USI trained weights. Add an alternate downsample mode to EdgeNeXt and train a small model.Two srelpos (shared relative position) models trained, and a medium w/ class token. These are closer to YOLO-v5+ backbone defs. CspNet refactored with dataclass config, simplified CrossStage3 ( cs3) option.Small ResNet defs added by request with 1 block repeats for both basic and bottleneck (resnet10 and resnet14).Official research models (w/ weights) added:.Version 0.6.7 PyPi release (/w above bug fixes and new weighs since 0.6.5).deit3 models not being able to resize pos_emb fixed.Add output_stride=8 and 16 support to ConvNeXt (dilation).cs3* weights above all trained on TPU w/ bits_and_tpu branch.All runtime benchmark and validation result csv files are finally up-to-date!.Add freshly minted DeiT-III Medium (width=512, depth=12, num_heads=8) model weights.Updated EdgeNeXt to improve ONNX export, add new base variant and weights from original ( ). More custom ConvNeXt smaller model defs with weights.'Fast Norm' support for LayerNorm and GroupNorm that avoids float32 upcast w/ AMP (uses APEX LN if available for further boost).PyramidVisionTransformer-V2 (adapted from ).MViT-V2 (multi-scale vit, adapted from ).GCVit (weights adapted from, code 100% timm re-write for license purposes).(T) = TPU trained with bits_and_tpu branch training code, (G) = GPU trained.coatnet_0_rw_224 - 82.4 (T) - NOTE timm '0' coatnets have 2 more 3rd stage blocks.Initial CoAtNet and MaxVit timm pretrained weights (working on more):.an unfinished Tensorflow version from MaxVit authors can be found.both found in maxxvit.py model def, contains numerous experiments outside scope of original papers.CoAtNet ( ) and MaxVit ( ) timm original models.Add new RelPosMlp MaxViT weight that leverages this: MaxVit window size scales with img_size by default.Add more weights in maxxvit series incl a pico (7.5M params, 1.9 GMACs), two tiny variants:.Add BEiT-v2 weights for base and large 224x224 models from.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |