How to use twhitworth/KAT-Dev-72B-Exp-AWQ-INT4-noct with Transformers:
# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("twhitworth/KAT-Dev-72B-Exp-AWQ-INT4-noct") model = AutoModelForCausalLM.from_pretrained("twhitworth/KAT-Dev-72B-Exp-AWQ-INT4-noct", device_map="auto")
twhitworth