tag:blogger.com,1999:blog-34962591571301846602024-03-18T02:48:06.726-07:00Fahd Mirza on AI, Cloud, DevOps and DatabasesBlog By Fahd Mirza ChughtaiFahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.comBlogger513125tag:blogger.com,1999:blog-3496259157130184660.post-57340165078993776702024-03-15T19:54:00.000-07:002024-03-15T19:58:00.854-07:00How to Create Urdu Hindi AI Model and Dataset from New Dataset<p> This video is hands on step-by-step tutorial to create a new dataset, an AI model, fine-tune the model on dataset and then push it to hugging face.</p><span><a name='more'></a></span><p><br /></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="348" src="https://www.youtube.com/embed/tjahjuROCWk" width="476" youtube-src-id="tjahjuROCWk"></iframe></div><br /><p><br /></p><p><b>Code:</b></p><p>%%capture</p><p>import torch</p><p>major_version, minor_version = torch.cuda.get_device_capability()</p><p># Must install separately since Colab has torch 2.2.1, which breaks packages</p><p>!pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"</p><p>if major_version >= 8:</p><p> # Use this for new GPUs like Ampere, Hopper GPUs (RTX 30xx, RTX 40xx, A100, H100, L40)</p><p> !pip install --no-deps packaging ninja flash-attn xformers trl peft accelerate bitsandbytes</p><p>else:</p><p> # Use this for older GPUs (V100, Tesla T4, RTX 20xx)</p><p> !pip install --no-deps xformers trl peft accelerate bitsandbytes</p><p>pass</p><p><br /></p><p>!pip install einops</p><p><br /></p><p>from unsloth import FastLanguageModel</p><p>import torch</p><p>max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!</p><p>dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+</p><p>load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.</p><p><br /></p><p>model, tokenizer = FastLanguageModel.from_pretrained(</p><p> model_name = "unsloth/gemma-7b-bnb-4bit", # Choose ANY! eg teknium/OpenHermes-2.5-Mistral-7B</p><p> max_seq_length = max_seq_length,</p><p> dtype = dtype,</p><p> load_in_4bit = load_in_4bit,</p><p> token = " ", # use one if using gated models like meta-llama/Llama-2-7b-hf</p><p>)</p><p><br /></p><p>model = FastLanguageModel.get_peft_model(</p><p> model,</p><p> r = 16, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128</p><p> target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",</p><p> "gate_proj", "up_proj", "down_proj",],</p><p> lora_alpha = 16,</p><p> lora_dropout = 0, # Supports any, but = 0 is optimized</p><p> bias = "none", # Supports any, but = "none" is optimized</p><p> use_gradient_checkpointing = True,</p><p> random_state = 3407,</p><p> use_rslora = False, # We support rank stabilized LoRA</p><p> loftq_config = None, # And LoftQ</p><p>)</p><p><br /></p><p>alpaca_prompt = """ذیل میں ایک ہدایت ہے جو فلم کے نام کی وضاحت کرتی ہے، اس کے ساتھ ایک ان پٹ بھی ہے جو مزید دستاویزات فراہم کرتا ہے۔ گانے کے بول لکھنے کے لیے ایک لمحہ نکالیں جو فلم کے نام کے معنی سے میل کھاتا ہے۔</p><p><br /></p><p>### Instruction:</p><p>{}</p><p><br /></p><p>### Input:</p><p>{}</p><p><br /></p><p>### Response:</p><p>{}"""</p><p><br /></p><p>EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN</p><p>def formatting_prompts_func(examples):</p><p> instructions = examples["urdu_instruction"]</p><p> inputs = examples["urdu_input"]</p><p> outputs = examples["urdu_output"]</p><p> texts = []</p><p> for instruction, input, output in zip(instructions, inputs, outputs):</p><p> # Must add EOS_TOKEN, otherwise your generation will go on forever!</p><p> text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN</p><p> texts.append(text)</p><p> return { "text" : texts, }</p><p>pass</p><p><br /></p><p>from datasets import load_dataset</p><p>dataset = load_dataset("fahdmirzac/urdu_bollywood_songs_dataset", split = "train")</p><p>dataset = dataset.map(formatting_prompts_func, batched = True,)</p><p><br /></p><p>from huggingface_hub import login</p><p>access_token = "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"</p><p>login(token=access_token)</p><p><br /></p><p>from trl import SFTTrainer</p><p>from transformers import TrainingArguments</p><p><br /></p><p>trainer = SFTTrainer(</p><p> model = model,</p><p> tokenizer = tokenizer,</p><p> train_dataset = dataset,</p><p> dataset_text_field = "text",</p><p> max_seq_length = max_seq_length,</p><p> dataset_num_proc = 2,</p><p> packing = False, # Can make training 5x faster for short sequences.</p><p> args = TrainingArguments(</p><p> per_device_train_batch_size = 2,</p><p> gradient_accumulation_steps = 4,</p><p> warmup_steps = 5,</p><p> max_steps = 100,</p><p> learning_rate = 2e-4,</p><p> fp16 = not torch.cuda.is_bf16_supported(),</p><p> bf16 = torch.cuda.is_bf16_supported(),</p><p> logging_steps = 1,</p><p> optim = "adamw_8bit",</p><p> weight_decay = 0.01,</p><p> lr_scheduler_type = "linear",</p><p> seed = 3407,</p><p> output_dir = "outputs",</p><p> ),</p><p>)</p><p><br /></p><p>trainer_stats = trainer.train()</p><p><br /></p><p>FastLanguageModel.for_inference(model) # Enable native 2x faster inference</p><p>inputs = tokenizer(</p><p>[</p><p> alpaca_prompt.format(</p><p> "دیے گئے فلم کے نام کے بارے میں ایک مختصر گیت کے بول لکھیں۔", # instruction</p><p> "کیوں پیار ہو گیا", # input</p><p> "", # output - leave this blank for generation!</p><p> )</p><p>], return_tensors = "pt").to("cuda")</p><p><br /></p><p>outputs = model.generate(**inputs, max_new_tokens = 200, use_cache = True)</p><p>tokenizer.batch_decode(outputs)</p><p><br /></p><p>FastLanguageModel.for_inference(model) # Enable native 2x faster inference</p><p>inputs = tokenizer(</p><p>[</p><p> alpaca_prompt.format(</p><p> "دیے گئے فلم کے نام کے بارے میں ایک مختصر گیت کے بول لکھیں۔", # instruction</p><p> "رنگ", # input</p><p> "", # output - leave this blank for generation!</p><p> )</p><p>], return_tensors = "pt").to("cuda")</p><p><br /></p><p>outputs = model.generate(**inputs, max_new_tokens = 200, use_cache = True)</p><p>tokenizer.batch_decode(outputs)</p><p><br /></p><p>model.push_to_hub("fahdmirzac/Gemma_Urdu_Hindi_Bollywood_Songs", token = "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")</p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-29175137790936812622024-03-15T00:58:00.000-07:002024-03-15T00:58:31.681-07:00Using Claude 3 Haiku Vision with Amazon Bedrock Locally<p> This video is a hands-on guide as how to use vision features of Anthropic's Claude 3 Haiku AI model with Amazon Bedrock.</p><span><a name='more'></a></span><p></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="324" src="https://www.youtube.com/embed/8I8r_nH8T1c" width="490" youtube-src-id="8I8r_nH8T1c"></iframe></div><br /><b><br /></b><p></p><p><b>Code Used:</b></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 20px; line-height: 27px; white-space: pre;"><div><span style="color: #c586c0;">import</span> boto3</div><div><span style="color: #c586c0;">import</span> json</div><div><span style="color: #c586c0;">import</span> base64</div><div><span style="color: #c586c0;">from</span> botocore.exceptions <span style="color: #c586c0;">import</span> ClientError</div><br /><div>bedrock <span style="color: #d4d4d4;">=</span> boto3.client(<span style="color: #9cdcfe;">service_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">"bedrock-runtime"</span>,<span style="color: #9cdcfe;">region_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'us-east-1'</span>)</div><br /><div>modelId <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">"anthropic.claude-3-haiku-20240307-v1:0"</span></div><br /><div>accept <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">"application/json"</span></div><div>contentType <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">"application/json"</span></div><br /><br /><div><span style="color: #6a9955;"># prompt = "What is written in this image?"</span></div><div><span style="color: #6a9955;"># image_path = "./images/ab55.png"</span></div><br /><div><span style="color: #6a9955;"># prompt = "How many faces are there in this image and what are the expressions of those faces?"</span></div><div><span style="color: #6a9955;"># image_path = "./images/expression.png"</span></div><br /><div><span style="color: #6a9955;"># prompt = "Tell me a short story about this image."</span></div><div><span style="color: #6a9955;"># image_path = "./images/hiking.png"</span></div><br /><div>prompt <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">"What's the location in this image?"</span></div><div>image_path <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">"./images/whereisthis.png"</span></div><br /><br /><div><span style="color: #c586c0;">with</span> <span style="color: #dcdcaa;">open</span>(image_path, <span style="color: #ce9178;">"rb"</span>) <span style="color: #c586c0;">as</span> image_file:</div><div> image <span style="color: #d4d4d4;">=</span> base64.b64encode(image_file.read()).decode(<span style="color: #ce9178;">"utf8"</span>)</div><br /><div>request_body <span style="color: #d4d4d4;">=</span> {</div><div> <span style="color: #ce9178;">"anthropic_version"</span>: <span style="color: #ce9178;">"bedrock-2023-05-31"</span>,</div><div> <span style="color: #ce9178;">"max_tokens"</span>: <span style="color: #b5cea8;">2048</span>,</div><div> <span style="color: #ce9178;">"messages"</span>: [</div><div> {</div><div> <span style="color: #ce9178;">"role"</span>: <span style="color: #ce9178;">"user"</span>,</div><div> <span style="color: #ce9178;">"content"</span>: [</div><div> {</div><div> <span style="color: #ce9178;">"type"</span>: <span style="color: #ce9178;">"text"</span>,</div><div> <span style="color: #ce9178;">"text"</span>: prompt,</div><div> },</div><div> {</div><div> <span style="color: #ce9178;">"type"</span>: <span style="color: #ce9178;">"image"</span>,</div><div> <span style="color: #ce9178;">"source"</span>: {</div><div> <span style="color: #ce9178;">"type"</span>: <span style="color: #ce9178;">"base64"</span>,</div><div> <span style="color: #ce9178;">"media_type"</span>: <span style="color: #ce9178;">"image/png"</span>,</div><div> <span style="color: #ce9178;">"data"</span>: image,</div><div> },</div><div> },</div><div> ],</div><div> }</div><div> ],</div><div>}</div><br /><div><span style="color: #c586c0;">try</span>:</div><div> response <span style="color: #d4d4d4;">=</span> bedrock.invoke_model(</div><div> <span style="color: #9cdcfe;">modelId</span><span style="color: #d4d4d4;">=</span>modelId,</div><div> <span style="color: #9cdcfe;">body</span><span style="color: #d4d4d4;">=</span>json.dumps(request_body),</div><div> )</div><br /><div> <span style="color: #6a9955;"># Process and print the response</span></div><div> result <span style="color: #d4d4d4;">=</span> json.loads(response.get(<span style="color: #ce9178;">"body"</span>).read())</div><div> input_tokens <span style="color: #d4d4d4;">=</span> result[<span style="color: #ce9178;">"usage"</span>][<span style="color: #ce9178;">"input_tokens"</span>]</div><div> output_tokens <span style="color: #d4d4d4;">=</span> result[<span style="color: #ce9178;">"usage"</span>][<span style="color: #ce9178;">"output_tokens"</span>]</div><div> output_list <span style="color: #d4d4d4;">=</span> result.get(<span style="color: #ce9178;">"content"</span>, [])</div><br /><div> <span style="color: #6a9955;"># print("Invocation details:")</span></div><div> <span style="color: #6a9955;"># print(f"- The input length is {input_tokens} tokens.")</span></div><div> <span style="color: #6a9955;"># print(f"- The output length is {output_tokens} tokens.")</span></div><br /><div> <span style="color: #6a9955;"># print(f"- The model returned {len(output_list)} response(s):")</span></div><div> <span style="color: #c586c0;">for</span> output <span style="color: #c586c0;">in</span> output_list:</div><div> <span style="color: #dcdcaa;">print</span>(output[<span style="color: #ce9178;">"text"</span>])</div><br /><div><span style="color: #c586c0;">except</span> ClientError <span style="color: #c586c0;">as</span> err:</div><div> <span style="color: #dcdcaa;">print</span>(</div><div> <span style="color: #ce9178;">"Couldn't invoke Claude 3 Haiku Vision. Here's why: </span><span style="color: #569cd6;">%s</span><span style="color: #ce9178;">: </span><span style="color: #569cd6;">%s</span><span style="color: #ce9178;">"</span>,</div><div> err.response[<span style="color: #ce9178;">"Error"</span>][<span style="color: #ce9178;">"Code"</span>],</div><div> err.response[<span style="color: #ce9178;">"Error"</span>][<span style="color: #ce9178;">"Message"</span>],</div><div> )</div><div> <span style="color: #c586c0;">raise</span></div></div>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-87710622019438047982024-03-14T20:03:00.000-07:002024-03-14T20:08:52.475-07:00Create AI Agent in AWS with Boto3 Code <p> This video is a step-by-step tutorial with code as how to create Amazon Bedrock AI agents with boto3 in Python to integrate with Lambda.</p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="366" src="https://www.youtube.com/embed/RHWwy2QD7Xo" width="493" youtube-src-id="RHWwy2QD7Xo"></iframe></div><br /><p><br /></p><p>Code used: Just use any lambda with it of your choice.</p><p><br /></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 20px; line-height: 27px; white-space: pre;"><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">logging</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">boto3</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">time</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">yaml</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">json</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">io</span> </div><div><span style="color: #c586c0;">from</span> <span style="color: #4ec9b0;">botocore</span>.<span style="color: #4ec9b0;">exceptions</span> <span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">ClientError</span></div><br /><div><span style="color: #569cd6;">def</span> <span style="color: #dcdcaa;">create_agent</span>(<span style="color: #9cdcfe;">bedrock</span>, <span style="color: #9cdcfe;">agent_name</span>, <span style="color: #9cdcfe;">foundation_model</span>, <span style="color: #9cdcfe;">role_arn</span>, <span style="color: #9cdcfe;">instruction</span>):</div><div> <span style="color: #c586c0;">try</span>:</div><div> <span style="color: #6a9955;"># Create a low-level client with the service name</span></div><div> <span style="color: #9cdcfe;">response</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">bedrock</span>.create_agent(</div><div> <span style="color: #9cdcfe;">agentName</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">agent_name</span>,</div><div> <span style="color: #9cdcfe;">foundationModel</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">foundation_model</span>,</div><div> <span style="color: #9cdcfe;">agentResourceRoleArn</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">role_arn</span>,</div><div> <span style="color: #9cdcfe;">instruction</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">instruction</span>,</div><div> )</div><div> <span style="color: #c586c0;">except</span> <span style="color: #4ec9b0;">ClientError</span> <span style="color: #c586c0;">as</span> <span style="color: #9cdcfe;">e</span>:</div><div> <span style="color: #4ec9b0;">logging</span>.<span style="color: #dcdcaa;">error</span>(<span style="color: #569cd6;">f</span><span style="color: #ce9178;">"Couldn't create agent due to: </span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">e</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">"</span>)</div><div> <span style="color: #c586c0;">raise</span></div><div> <span style="color: #c586c0;">else</span>:</div><div> <span style="color: #c586c0;">return</span> <span style="color: #9cdcfe;">response</span>[<span style="color: #ce9178;">"agent"</span>]</div><br /><div><span style="color: #569cd6;">def</span> <span style="color: #dcdcaa;">create_agent_action_group</span>(<span style="color: #9cdcfe;">bedrock</span>, <span style="color: #9cdcfe;">name</span>, <span style="color: #9cdcfe;">description</span>, <span style="color: #9cdcfe;">agent_id</span>, <span style="color: #9cdcfe;">agent_version</span>, <span style="color: #9cdcfe;">function_arn</span>, <span style="color: #9cdcfe;">api_schema</span>):</div><div> <span style="color: #c586c0;">try</span>:</div><div> <span style="color: #9cdcfe;">response</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">bedrock</span>.create_agent_action_group(</div><div> <span style="color: #9cdcfe;">actionGroupName</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">name</span>,</div><div> <span style="color: #9cdcfe;">description</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">description</span>,</div><div> <span style="color: #9cdcfe;">agentId</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">agent_id</span>,</div><div> <span style="color: #9cdcfe;">agentVersion</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">agent_version</span>,</div><div> <span style="color: #9cdcfe;">actionGroupExecutor</span><span style="color: #d4d4d4;">=</span>{<span style="color: #ce9178;">"lambda"</span>: <span style="color: #9cdcfe;">function_arn</span>},</div><div> <span style="color: #9cdcfe;">apiSchema</span><span style="color: #d4d4d4;">=</span>{<span style="color: #ce9178;">"payload"</span>: <span style="color: #9cdcfe;">api_schema</span>},</div><div> )</div><div> <span style="color: #9cdcfe;">agent_action_group</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">response</span>[<span style="color: #ce9178;">"agentActionGroup"</span>]</div><div> <span style="color: #c586c0;">except</span> <span style="color: #4ec9b0;">ClientError</span> <span style="color: #c586c0;">as</span> <span style="color: #9cdcfe;">e</span>:</div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #569cd6;">f</span><span style="color: #ce9178;">"Error: Couldn't create agent action group. Here's why: </span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">e</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">"</span>)</div><div> <span style="color: #c586c0;">raise</span></div><div> <span style="color: #c586c0;">else</span>:</div><div> <span style="color: #c586c0;">return</span> <span style="color: #9cdcfe;">agent_action_group</span></div><br /><div><span style="color: #569cd6;">def</span> <span style="color: #dcdcaa;">prepare_agent</span>(<span style="color: #9cdcfe;">bedrock</span>, <span style="color: #9cdcfe;">agent_id</span>):</div><div> <span style="color: #c586c0;">try</span>:</div><div> <span style="color: #9cdcfe;">prepared_agent_details</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">bedrock</span>.prepare_agent(<span style="color: #9cdcfe;">agentId</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">agent_id</span>)</div><div> <span style="color: #c586c0;">except</span> <span style="color: #4ec9b0;">ClientError</span> <span style="color: #c586c0;">as</span> <span style="color: #9cdcfe;">e</span>:</div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #569cd6;">f</span><span style="color: #ce9178;">"Couldn't prepare agent. </span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">e</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">"</span>)</div><div> <span style="color: #c586c0;">raise</span></div><div> <span style="color: #c586c0;">else</span>:</div><div> <span style="color: #c586c0;">return</span> <span style="color: #9cdcfe;">prepared_agent_details</span></div><br /><div><span style="color: #569cd6;">def</span> <span style="color: #dcdcaa;">create_agent_alias</span>(<span style="color: #9cdcfe;">bedrock</span>, <span style="color: #9cdcfe;">name</span>, <span style="color: #9cdcfe;">agent_id</span>):</div><div> <span style="color: #c586c0;">try</span>:</div><div> <span style="color: #9cdcfe;">response</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">bedrock</span>.create_agent_alias(</div><div> <span style="color: #9cdcfe;">agentAliasName</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">name</span>, <span style="color: #9cdcfe;">agentId</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">agent_id</span></div><div> )</div><div> <span style="color: #9cdcfe;">agent_alias</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">response</span>[<span style="color: #ce9178;">"agentAlias"</span>]</div><div> <span style="color: #c586c0;">except</span> <span style="color: #4ec9b0;">ClientError</span> <span style="color: #c586c0;">as</span> <span style="color: #9cdcfe;">e</span>:</div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #569cd6;">f</span><span style="color: #ce9178;">"Couldn't create agent alias. </span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">e</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">"</span>)</div><div> <span style="color: #c586c0;">raise</span></div><div> <span style="color: #c586c0;">else</span>:</div><div> <span style="color: #c586c0;">return</span> <span style="color: #9cdcfe;">agent_alias</span></div><br /><br /><br /><div><span style="color: #569cd6;">def</span> <span style="color: #dcdcaa;">main</span>():</div><div> <span style="color: #6a9955;"># Define your parameters</span></div><div> <span style="color: #9cdcfe;">bedrock</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">boto3</span>.<span style="color: #dcdcaa;">client</span>(<span style="color: #9cdcfe;">service_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'bedrock-agent'</span>,<span style="color: #9cdcfe;">region_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'us-east-1'</span>)</div><div> <span style="color: #9cdcfe;">agent_name</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">'AstroAI'</span></div><div> <span style="color: #9cdcfe;">foundation_model</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">'anthropic.claude-v2'</span></div><div> <span style="color: #9cdcfe;">role_arn</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">'bedrock role arn'</span></div><div> <span style="color: #9cdcfe;">instruction</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">'Your task is to generate unique and insightful daily horoscopes for individuals </span><span style="color: #569cd6;">\</span></div><div><span style="color: #ce9178;"> based on their zodiac sign. Start by analyzing the general characteristics and common </span><span style="color: #569cd6;">\</span></div><div><span style="color: #ce9178;"> themes associated with each zodiac sign. Consider traits, challenges, opportunities, </span><span style="color: #569cd6;">\</span></div><div><span style="color: #ce9178;"> and the emotional and physical wellbeing of individuals under each sign. Use this </span><span style="color: #569cd6;">\</span></div><div><span style="color: #ce9178;"> understanding to create personalized, relevant, and engaging horoscopes that offer </span><span style="color: #569cd6;">\</span></div><div><span style="color: #ce9178;"> guidance, reflection, and encouragement for the day ahead. Ensure the horoscopes </span><span style="color: #569cd6;">\</span></div><div><span style="color: #ce9178;"> are varied and resonate with the unique qualities of each sign, contributing </span><span style="color: #569cd6;">\</span></div><div><span style="color: #ce9178;"> positively to the users day.'</span></div><br /><div> <span style="color: #6a9955;"># Call the create_agent function</span></div><div> <span style="color: #c586c0;">try</span>:</div><div> <span style="color: #9cdcfe;">agent</span> <span style="color: #d4d4d4;">=</span> <span style="color: #dcdcaa;">create_agent</span>(<span style="color: #9cdcfe;">bedrock</span>, <span style="color: #9cdcfe;">agent_name</span>, <span style="color: #9cdcfe;">foundation_model</span>, <span style="color: #9cdcfe;">role_arn</span>, <span style="color: #9cdcfe;">instruction</span>)</div><div> <span style="color: #9cdcfe;">agent_id</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">agent</span>[<span style="color: #ce9178;">'agentId'</span>]</div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #569cd6;">f</span><span style="color: #ce9178;">"Agent created successfully: </span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">agent_id</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">"</span>)</div><div> <span style="color: #c586c0;">except</span> <span style="color: #4ec9b0;">ClientError</span>:</div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #ce9178;">"Failed to create the agent."</span>)</div><br /><div> <span style="color: #4ec9b0;">time</span>.<span style="color: #dcdcaa;">sleep</span>(<span style="color: #b5cea8;">10</span>)</div><br /><div> <span style="color: #c586c0;">try</span>:</div><div> <span style="color: #c586c0;">with</span> <span style="color: #dcdcaa;">open</span>(<span style="color: #ce9178;">"api_schema.yaml"</span>) <span style="color: #c586c0;">as</span> <span style="color: #9cdcfe;">file</span>:</div><div> <span style="color: #9cdcfe;">api_schema</span><span style="color: #d4d4d4;">=</span><span style="color: #4ec9b0;">json</span>.<span style="color: #dcdcaa;">dumps</span>(<span style="color: #4ec9b0;">yaml</span>.<span style="color: #dcdcaa;">safe_load</span>(<span style="color: #9cdcfe;">file</span>))</div><div> <span style="color: #9cdcfe;">name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">"AstroGroup"</span></div><div> <span style="color: #9cdcfe;">description</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">"AI Astrologer"</span></div><div> <span style="color: #9cdcfe;">agent_version</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">"DRAFT"</span></div><div> <span style="color: #9cdcfe;">function_arn</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">"arn:aws:lambda:us-east-1::function:horoscope"</span></div><div> <span style="color: #9cdcfe;">agentgroup</span> <span style="color: #d4d4d4;">=</span> <span style="color: #dcdcaa;">create_agent_action_group</span>(<span style="color: #9cdcfe;">bedrock</span>, <span style="color: #9cdcfe;">name</span>, <span style="color: #9cdcfe;">description</span>, <span style="color: #9cdcfe;">agent_id</span>, <span style="color: #9cdcfe;">agent_version</span>, <span style="color: #9cdcfe;">function_arn</span>, <span style="color: #9cdcfe;">api_schema</span>) </div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #9cdcfe;">agentgroup</span>[<span style="color: #ce9178;">'actionGroupId'</span>])</div><div> <span style="color: #c586c0;">except</span> <span style="color: #4ec9b0;">ClientError</span> <span style="color: #c586c0;">as</span> <span style="color: #9cdcfe;">e</span>:</div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #569cd6;">f</span><span style="color: #ce9178;">"Couldn't create agent action group. Here's why: </span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">e</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">"</span>)</div><div> <span style="color: #c586c0;">raise</span> </div><br /><div> <span style="color: #4ec9b0;">time</span>.<span style="color: #dcdcaa;">sleep</span>(<span style="color: #b5cea8;">5</span>)</div><br /><div> <span style="color: #9cdcfe;">agentprepared</span> <span style="color: #d4d4d4;">=</span> <span style="color: #dcdcaa;">prepare_agent</span>(<span style="color: #9cdcfe;">bedrock</span>, <span style="color: #9cdcfe;">agent_id</span>) </div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #9cdcfe;">agentprepared</span>)</div><br /><div> <span style="color: #4ec9b0;">time</span>.<span style="color: #dcdcaa;">sleep</span>(<span style="color: #b5cea8;">20</span>)</div><br /><div> <span style="color: #9cdcfe;">agentalias</span> <span style="color: #d4d4d4;">=</span> <span style="color: #dcdcaa;">create_agent_alias</span>(<span style="color: #9cdcfe;">bedrock</span>, <span style="color: #9cdcfe;">name</span>, <span style="color: #9cdcfe;">agent_id</span>)</div><div> <span style="color: #dcdcaa;">print</span>(<span style="color: #9cdcfe;">agentalias</span>[<span style="color: #ce9178;">'agentAliasId'</span>])</div><br /><div><span style="color: #c586c0;">if</span> <span style="color: #9cdcfe;">__name__</span> <span style="color: #d4d4d4;">==</span> <span style="color: #ce9178;">"__main__"</span>:</div><div> <span style="color: #dcdcaa;">main</span>()</div><br /></div>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-49825459068106248502024-02-28T23:47:00.000-08:002024-02-28T23:47:51.781-08:00How AI Transcription Can Make Your Next Job Interview More Effective<p> If you are looking to use AI Transcription during your job interview to get help from LLM, then this video is going to help you immensely.</p><span><a name='more'></a></span><p><br /></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="361" src="https://www.youtube.com/embed/STMFhT4aAv8" width="487" youtube-src-id="STMFhT4aAv8"></iframe></div><br /><p><br /></p><p><br /></p><p style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; margin: 1.25em 0px; white-space-collapse: preserve;">In the rapidly evolving landscape of recruitment, the integration of Artificial Intelligence (AI) tools has been a game-changer, especially AI transcription services. These services, which convert speech into a written or electronic text document, are now being increasingly utilized during candidate interviews. This innovative approach is not just about efficiency; it's about transforming the interview process into a more insightful, fair, and strategic component of talent acquisition. Let's delve into the merits of using AI transcription in candidate interviews and how it is reshaping the future of hiring.</p><h2 style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 1.33333; margin: 2rem 0px 1rem; white-space-collapse: preserve;">Enhanced Accuracy and Objectivity</h2><p style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; margin: 0px 0px 1.25em; white-space-collapse: preserve;">One of the paramount benefits of using AI transcription is the enhanced accuracy and objectivity it brings to the recruitment process. Traditional note-taking methods during interviews are inherently flawed, prone to human error, and can be biased. AI transcription, on the other hand, captures every word as it is spoken, ensuring that nothing is missed or misinterpreted. This verbatim record allows hiring managers to review the interview with a fine-tooth comb, ensuring that decisions are made based on what was actually said, rather than on recollection or interpretation.</p><h2 style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 1.33333; margin: 2rem 0px 1rem; white-space-collapse: preserve;">Improved Candidate Experience</h2><p style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; margin: 0px 0px 1.25em; white-space-collapse: preserve;">The candidate experience is crucial in attracting top talent. AI transcription contributes to a more engaging and less intimidating interview environment. Candidates can speak freely, knowing that their responses are being accurately recorded. This level of precision in capturing responses allows candidates to feel heard and valued, improving their overall experience and perception of the company. Furthermore, the efficiency of AI transcription speeds up the interview process, enabling quicker feedback and follow-ups, which candidates appreciate.</p><h2 style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 1.33333; margin: 2rem 0px 1rem; white-space-collapse: preserve;">Accessibility and Inclusivity</h2><p style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; margin: 0px 0px 1.25em; white-space-collapse: preserve;">AI transcription makes interviews more accessible and inclusive, breaking down barriers for candidates who may be deaf or hard of hearing. By providing real-time transcription, these candidates can participate fully in the interview process, ensuring equal opportunities for all. This commitment to inclusivity not only enhances the company's reputation but also widens the talent pool by making it more diverse.</p><h2 style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 1.33333; margin: 2rem 0px 1rem; white-space-collapse: preserve;">Streamlined Analysis and Decision Making</h2><p style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; margin: 0px 0px 1.25em; white-space-collapse: preserve;">The comprehensive data captured by AI transcription allows for a more streamlined analysis of candidates' responses. Hiring teams can utilize this data to identify patterns, evaluate key competencies, and make more informed decisions. AI tools can also highlight specific skills and qualifications mentioned during the interview, making it easier to assess candidates against job requirements. This data-driven approach reduces the likelihood of cognitive biases affecting hiring decisions and helps in selecting the best fit for the role.</p><h2 style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 1.33333; margin: 2rem 0px 1rem; white-space-collapse: preserve;">Enhanced Collaboration Among Hiring Teams</h2><p style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; margin: 0px 0px 1.25em; white-space-collapse: preserve;">AI transcription facilitates better collaboration among hiring team members, especially in scenarios where not all members can attend every interview. The transcribed interviews can be easily shared, allowing team members to review and provide input at their convenience. This collaborative approach ensures a more comprehensive evaluation of candidates, drawing on diverse perspectives within the team.</p><h2 style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 1.33333; margin: 2rem 0px 1rem; white-space-collapse: preserve;">Legal Compliance and Record-Keeping</h2><p style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; margin: 0px 0px 1.25em; white-space-collapse: preserve;">In today's litigious environment, maintaining accurate records of candidate interviews can be crucial for legal compliance. AI transcription provides an exact record of the conversation, which can be invaluable in defending against claims of discrimination or bias. This level of documentation ensures that the hiring process is transparent and defensible, safeguarding the company against potential legal challenges.</p><h2 style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; line-height: 1.33333; margin: 2rem 0px 1rem; white-space-collapse: preserve;">Conclusion</h2><p style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgba(69,89,164,.5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 transparent; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 transparent; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 transparent; --tw-shadow: 0 0 transparent; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #212121; border: 0px solid rgb(227, 227, 227); box-sizing: border-box; color: #ececec; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; font-size: 16px; margin: 0px; white-space-collapse: preserve;">The adoption of AI transcription in candidate interviews is more than just a nod to technological advancement; it's a strategic move towards making the recruitment process more accurate, fair, and inclusive. By enhancing the accuracy of interviews, improving candidate experience, and streamlining decision-making, AI transcription is setting a new standard in talent acquisition. As companies continue to seek competitive advantages in attracting top talent, the use of AI in recruitment processes, especially transcription, is likely to become not just common but essential.</p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-17570003692657099202024-02-26T18:20:00.000-08:002024-02-26T18:21:29.881-08:00Tutorial to Implement RAG with Gemma Model Locally<p> Unlock the power of Retrieval-Augmented Generation (RAG) locally using the Gemma model with our detailed step-by-step tutorial. Learn how to enhance your projects by integrating RAG for insightful document processing and AI-driven content generation. Perfect for developers, data scientists, and AI enthusiasts eager to leverage advanced NLP techniques on their own documents. No prior RAG experience required!</p><span><a name='more'></a></span><p><br /></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="344" src="https://www.youtube.com/embed/d-0r5F70rHk" width="466" youtube-src-id="d-0r5F70rHk"></iframe></div><br /><p><br /></p><p><b>Commands Used</b>:</p><p><br /></p><p>pip install -U "transformers==4.38.1" --upgrade</p><p>pip install -q pypdf</p><p>pip install -q python-dotenv</p><p>pip install llama-index==0.10.12</p><p>pip install -q gradio</p><p>pip install einops</p><p>pip install accelerate</p><p>pip install llama-index-llms-huggingface</p><p>pip install llama-index-embeddings-fastembed</p><p>pip install fastembed</p><p><br /></p><p>import logging</p><p>import sys</p><p><br /></p><p>logging.basicConfig(stream=sys.stdout, level=logging.INFO)</p><p>logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))</p><p><br /></p><p>from llama_index.core import VectorStoreIndex, SimpleDirectoryReader</p><p>from llama_index.llms.huggingface import HuggingFaceLLM</p><p>from llama_index.core import Settings</p><p><br /></p><p>documents = SimpleDirectoryReader("/home/ubuntu/pdfs").load_data()</p><p><br /></p><p>from llama_index.embeddings.fastembed import FastEmbedEmbedding</p><p><br /></p><p>embed_model = FastEmbedEmbedding(model_name="BAAI/bge-small-en-v1.5")</p><p>Settings.embed_model = embed_model</p><p>Settings.chunk_size = 512</p><p><br /></p><p>from llama_index.core import PromptTemplate</p><p><br /></p><p>system_prompt = "You are a Q&A assistant. Your goal is to answer questions as accurately as possible based on the instructions and context provided."</p><p><br /></p><p>query_wrapper_prompt = PromptTemplate("<|USER|>{query_str}<|ASSISTANT|>")</p><p><br /></p><p>from huggingface_hub.hf_api import HfFolder </p><p>HfFolder.save_token('<your huggingface token from huggingface.co>')</p><p><br /></p><p>import torch</p><p><br /></p><p>llm = HuggingFaceLLM(</p><p> context_window=8192,</p><p> max_new_tokens=256,</p><p> generate_kwargs={"temperature": 0.7, "do_sample": False},</p><p> system_prompt=system_prompt,</p><p> query_wrapper_prompt=query_wrapper_prompt,</p><p> tokenizer_name="google/gemma-7b-it",</p><p> model_name="google/gemma-7b-it",</p><p> device_map="auto",</p><p> tokenizer_kwargs={"max_length": 4096},</p><p> model_kwargs={"torch_dtype": torch.float16}</p><p>)</p><p><br /></p><p>Settings.llm = llm</p><p>Settings.chunk_size = 512</p><p><br /></p><p>index = VectorStoreIndex.from_documents(documents)</p><p><br /></p><p>query_engine = index.as_query_engine()</p><p><br /></p><p>def predict(input, history):</p><p> response = query_engine.query(input)</p><p> return str(response)</p><p> </p><p>import gradio as gr</p><p><br /></p><p>gr.ChatInterface(predict).launch(share=True)</p><p><br /></p><p> </p><p><br /></p><p><br /></p><p><br /></p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-60786942983455094162024-02-23T20:45:00.000-08:002024-02-23T21:25:26.745-08:00Build Your Own Audio Transcription Tool with AI<p> In this video, you will learn how to deploy a LLM based application intro production by using Amazon Bedrock, Amazon Transcribe to summarize audio files with ASR model, Titan. </p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="342" src="https://www.youtube.com/embed/uSP8gDlPYZU" width="492" youtube-src-id="uSP8gDlPYZU"></iframe></div><br /><p><br /></p><p>Code:</p><p><br /></p><p><br /></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 20px; line-height: 27px; white-space: pre;"><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">boto3</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">json</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">uuid</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">time</span></div><div><span style="color: #c586c0;">from</span> <span style="color: #4ec9b0;">jinja2</span> <span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">Template</span></div><br /><div><span style="color: #9cdcfe;">bedrock_runtime</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">boto3</span>.<span style="color: #dcdcaa;">client</span>(<span style="color: #ce9178;">'bedrock-runtime'</span>, <span style="color: #9cdcfe;">region_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'us-east-1'</span>)</div><br /><div><span style="color: #9cdcfe;">s3_client</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">boto3</span>.<span style="color: #dcdcaa;">client</span>(<span style="color: #9cdcfe;">service_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'s3'</span>, <span style="color: #9cdcfe;">region_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'ap-southeast-2'</span>)</div><br /><div><span style="color: #9cdcfe;">transcribe_client</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">boto3</span>.<span style="color: #dcdcaa;">client</span>(<span style="color: #ce9178;">'transcribe'</span>, <span style="color: #9cdcfe;">region_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'ap-southeast-2'</span>)</div><br /><div><span style="color: #9cdcfe;">bucket_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'<replace your bucket name here>'</span></div><br /><div><span style="color: #6a9955;">#file_name = 'angry.mp3'</span></div><div><span style="color: #9cdcfe;">file_name</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">'happy.mp3'</span></div><br /><div><span style="color: #9cdcfe;">job_name</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">'transcription-job-'</span> <span style="color: #d4d4d4;">+</span> <span style="color: #4ec9b0;">str</span>(<span style="color: #4ec9b0;">uuid</span>.<span style="color: #dcdcaa;">uuid4</span>())</div><br /><div><span style="color: #9cdcfe;">response</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">transcribe_client</span>.start_transcription_job(</div><div> <span style="color: #9cdcfe;">TranscriptionJobName</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">job_name</span>,</div><div> <span style="color: #9cdcfe;">Media</span><span style="color: #d4d4d4;">=</span>{<span style="color: #ce9178;">'MediaFileUri'</span>: <span style="color: #569cd6;">f</span><span style="color: #ce9178;">'s3://</span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">bucket_name</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">/</span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">file_name</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">'</span>},</div><div> <span style="color: #9cdcfe;">MediaFormat</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'mp3'</span>,</div><div> <span style="color: #9cdcfe;">LanguageCode</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'en-US'</span>,</div><div> <span style="color: #9cdcfe;">OutputBucketName</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">bucket_name</span>,</div><div> <span style="color: #9cdcfe;">Settings</span><span style="color: #d4d4d4;">=</span>{</div><div> <span style="color: #ce9178;">'ShowSpeakerLabels'</span>: <span style="color: #569cd6;">True</span>,</div><div> <span style="color: #ce9178;">'MaxSpeakerLabels'</span>: <span style="color: #b5cea8;">2</span></div><div> }</div><div>)</div><br /><div><span style="color: #c586c0;">while</span> <span style="color: #569cd6;">True</span>:</div><div> <span style="color: #9cdcfe;">status</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">transcribe_client</span>.get_transcription_job(<span style="color: #9cdcfe;">TranscriptionJobName</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">job_name</span>)</div><div> <span style="color: #c586c0;">if</span> <span style="color: #9cdcfe;">status</span>[<span style="color: #ce9178;">'TranscriptionJob'</span>][<span style="color: #ce9178;">'TranscriptionJobStatus'</span>] <span style="color: #569cd6;">in</span> [<span style="color: #ce9178;">'COMPLETED'</span>, <span style="color: #ce9178;">'FAILED'</span>]:</div><div> <span style="color: #c586c0;">break</span></div><div> <span style="color: #4ec9b0;">time</span>.<span style="color: #dcdcaa;">sleep</span>(<span style="color: #b5cea8;">2</span>)</div><div><span style="color: #dcdcaa;">print</span>(<span style="color: #9cdcfe;">status</span>[<span style="color: #ce9178;">'TranscriptionJob'</span>][<span style="color: #ce9178;">'TranscriptionJobStatus'</span>])</div><br /><div><span style="color: #c586c0;">if</span> <span style="color: #9cdcfe;">status</span>[<span style="color: #ce9178;">'TranscriptionJob'</span>][<span style="color: #ce9178;">'TranscriptionJobStatus'</span>] <span style="color: #d4d4d4;">==</span> <span style="color: #ce9178;">'COMPLETED'</span>:</div><div> </div><div> <span style="color: #6a9955;"># Load the transcript from S3.</span></div><div> <span style="color: #9cdcfe;">transcript_key</span> <span style="color: #d4d4d4;">=</span> <span style="color: #569cd6;">f</span><span style="color: #ce9178;">"</span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">job_name</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">.json"</span></div><div> <span style="color: #9cdcfe;">transcript_obj</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">s3_client</span>.get_object(<span style="color: #9cdcfe;">Bucket</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">bucket_name</span>, <span style="color: #9cdcfe;">Key</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">transcript_key</span>)</div><div> <span style="color: #9cdcfe;">transcript_text</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">transcript_obj</span>[<span style="color: #ce9178;">'Body'</span>].read().decode(<span style="color: #ce9178;">'utf-8'</span>)</div><div> <span style="color: #9cdcfe;">transcript_json</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">json</span>.<span style="color: #dcdcaa;">loads</span>(<span style="color: #9cdcfe;">transcript_text</span>)</div><div> </div><div> <span style="color: #9cdcfe;">output_text</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">""</span></div><div> <span style="color: #9cdcfe;">current_speaker</span> <span style="color: #d4d4d4;">=</span> <span style="color: #569cd6;">None</span></div><div> </div><div> <span style="color: #9cdcfe;">items</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">transcript_json</span>[<span style="color: #ce9178;">'results'</span>][<span style="color: #ce9178;">'items'</span>]</div><div> </div><div> <span style="color: #c586c0;">for</span> <span style="color: #9cdcfe;">item</span> <span style="color: #c586c0;">in</span> <span style="color: #9cdcfe;">items</span>:</div><div> </div><div> <span style="color: #9cdcfe;">speaker_label</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">item</span>.get(<span style="color: #ce9178;">'speaker_label'</span>, <span style="color: #569cd6;">None</span>)</div><div> <span style="color: #9cdcfe;">content</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">item</span>[<span style="color: #ce9178;">'alternatives'</span>][<span style="color: #b5cea8;">0</span>][<span style="color: #ce9178;">'content'</span>]</div><div> </div><div> <span style="color: #6a9955;"># Start the line with the speaker label:</span></div><div> <span style="color: #c586c0;">if</span> <span style="color: #9cdcfe;">speaker_label</span> <span style="color: #569cd6;">is</span> <span style="color: #569cd6;">not</span> <span style="color: #569cd6;">None</span> <span style="color: #569cd6;">and</span> <span style="color: #9cdcfe;">speaker_label</span> <span style="color: #d4d4d4;">!=</span> <span style="color: #9cdcfe;">current_speaker</span>:</div><div> <span style="color: #9cdcfe;">current_speaker</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">speaker_label</span></div><div> <span style="color: #9cdcfe;">output_text</span> <span style="color: #d4d4d4;">+=</span> <span style="color: #569cd6;">f</span><span style="color: #ce9178;">"</span><span style="color: #d7ba7d;">\n</span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">current_speaker</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">: "</span></div><div> </div><div> <span style="color: #6a9955;"># Add the speech content:</span></div><div> <span style="color: #c586c0;">if</span> <span style="color: #9cdcfe;">item</span>[<span style="color: #ce9178;">'type'</span>] <span style="color: #d4d4d4;">==</span> <span style="color: #ce9178;">'punctuation'</span>:</div><div> <span style="color: #9cdcfe;">output_text</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">output_text</span>.<span style="color: #dcdcaa;">rstrip</span>()</div><div> </div><div> <span style="color: #9cdcfe;">output_text</span> <span style="color: #d4d4d4;">+=</span> <span style="color: #569cd6;">f</span><span style="color: #ce9178;">"</span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">content</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;"> "</span></div><div> </div><div> <span style="color: #6a9955;"># Save the transcript to a text file</span></div><div> <span style="color: #c586c0;">with</span> <span style="color: #dcdcaa;">open</span>(<span style="color: #569cd6;">f</span><span style="color: #ce9178;">'</span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">job_name</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">.txt'</span>, <span style="color: #ce9178;">'w'</span>) <span style="color: #c586c0;">as</span> <span style="color: #9cdcfe;">f</span>:</div><div> <span style="color: #9cdcfe;">f</span>.<span style="color: #dcdcaa;">write</span>(<span style="color: #9cdcfe;">output_text</span>)</div><br /><div><span style="color: #c586c0;">with</span> <span style="color: #dcdcaa;">open</span>(<span style="color: #569cd6;">f</span><span style="color: #ce9178;">'</span><span style="color: #569cd6;">{</span><span style="color: #9cdcfe;">job_name</span><span style="color: #569cd6;">}</span><span style="color: #ce9178;">.txt'</span>, <span style="color: #ce9178;">"r"</span>) <span style="color: #c586c0;">as</span> <span style="color: #9cdcfe;">file</span>:</div><div> <span style="color: #9cdcfe;">transcript</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">file</span>.<span style="color: #dcdcaa;">read</span>()</div><br /><div><span style="color: #9cdcfe;">template_string</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">""" I need to summarize a conversation. The transcript of the </span></div><div><span style="color: #ce9178;">conversation is between the <data> XML like tags.</span></div><br /><div><span style="color: #ce9178;"><data></span></div><div><span style="color: #569cd6;">{{</span><span style="color: #ce9178;">transcript</span><span style="color: #569cd6;">}}</span></div><div><span style="color: #ce9178;"></data></span></div><br /><div><span style="color: #ce9178;">The summary must contain a one word sentiment analysis, and </span></div><div><span style="color: #ce9178;">a list of issues, problems or causes of friction</span></div><div><span style="color: #ce9178;">during the conversation. The output must be provided in </span></div><div><span style="color: #ce9178;">JSON format shown in the following example. </span></div><br /><div><span style="color: #ce9178;">Example output:</span></div><div><span style="color: #ce9178;">{</span></div><div><span style="color: #ce9178;"> "sentiment": <sentiment>,</span></div><div><span style="color: #ce9178;"> "issues": [</span></div><div><span style="color: #ce9178;"> {</span></div><div><span style="color: #ce9178;"> "topic": <topic>,</span></div><div><span style="color: #ce9178;"> "summary": <issue_summary>,</span></div><div><span style="color: #ce9178;"> }</span></div><div><span style="color: #ce9178;"> ]</span></div><div><span style="color: #ce9178;">}</span></div><br /><div><span style="color: #ce9178;">Write the JSON output and nothing more.</span></div><br /><div><span style="color: #ce9178;">Here is the JSON output: """</span></div><br /><div><span style="color: #9cdcfe;">data</span> <span style="color: #d4d4d4;">=</span> {</div><div> <span style="color: #ce9178;">'transcript'</span> : <span style="color: #9cdcfe;">transcript</span></div><div>}</div><br /><div><span style="color: #9cdcfe;">template</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">Template</span>(<span style="color: #9cdcfe;">template_string</span>)</div><br /><div><span style="color: #9cdcfe;">prompt</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">template</span>.<span style="color: #dcdcaa;">render</span>(<span style="color: #9cdcfe;">data</span>)</div><br /><div><span style="color: #9cdcfe;">kwargs</span> <span style="color: #d4d4d4;">=</span> {</div><div> <span style="color: #ce9178;">"modelId"</span>: <span style="color: #ce9178;">"amazon.titan-text-express-v1"</span>,</div><div> <span style="color: #ce9178;">"contentType"</span>: <span style="color: #ce9178;">"application/json"</span>,</div><div> <span style="color: #ce9178;">"accept"</span>: <span style="color: #ce9178;">"*/*"</span>,</div><div> <span style="color: #ce9178;">"body"</span>: <span style="color: #4ec9b0;">json</span>.<span style="color: #dcdcaa;">dumps</span>(</div><div> {</div><div> <span style="color: #ce9178;">"inputText"</span>: <span style="color: #9cdcfe;">prompt</span>,</div><div> <span style="color: #ce9178;">"textGenerationConfig"</span>: {</div><div> <span style="color: #ce9178;">"maxTokenCount"</span>: <span style="color: #b5cea8;">512</span>,</div><div> <span style="color: #ce9178;">"temperature"</span>: <span style="color: #b5cea8;">0</span>,</div><div> <span style="color: #ce9178;">"topP"</span>: <span style="color: #b5cea8;">0.9</span></div><div> }</div><div> }</div><div> )</div><div>}</div><br /><div><span style="color: #9cdcfe;">response</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">bedrock_runtime</span>.invoke_model(<span style="color: #d4d4d4;">**</span><span style="color: #9cdcfe;">kwargs</span>)</div><br /><div><span style="color: #9cdcfe;">response_body</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">json</span>.<span style="color: #dcdcaa;">loads</span>(<span style="color: #9cdcfe;">response</span>.get(<span style="color: #ce9178;">'body'</span>).read())</div><div><span style="color: #9cdcfe;">generation</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">response_body</span>[<span style="color: #ce9178;">'results'</span>][<span style="color: #b5cea8;">0</span>][<span style="color: #ce9178;">'outputText'</span>]</div><br /><div><span style="color: #dcdcaa;">print</span>(<span style="color: #9cdcfe;">generation</span>)</div></div>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-10915507012309845252024-02-22T23:59:00.000-08:002024-02-23T00:03:10.092-08:00Manage and Run Gemma LLM with Keras Locally<p>This video shows how to install and manage Gemma LLM with Keras. Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models.</p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="383" src="https://www.youtube.com/embed/Jt0x3Hp1tW8" width="461" youtube-src-id="Jt0x3Hp1tW8"></iframe></div><br /><p><br /></p><p>Code:</p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: monospace, Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><span style="color: #82c6ff;">!</span>pip install keras --upgrade</div><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: monospace, Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div style="line-height: 19px;"><span style="color: #82c6ff;"><br /></span></div><div style="line-height: 19px;"><span style="color: #82c6ff;">!</span>pip install kaggle</div><div style="line-height: 19px;"><div style="line-height: 19px;"><div><span style="color: #c586c0;"><br /></span></div><div><span style="color: #c586c0;">from</span> google.colab <span style="color: #c586c0;">import</span> files</div><br /><div>uploaded = files.upload<span style="color: gainsboro;">()</span></div><br /><div><span style="color: #c586c0;">for</span> fn <span style="color: #82c6ff;">in</span> uploaded.keys<span style="color: gainsboro;">():</span></div><div> <span style="color: #dcdcaa;">print</span><span style="color: gainsboro;">(</span><span style="color: #ce9178;">'User uploaded file "{name}" with length {length} bytes'</span>.<span style="color: #dcdcaa;">format</span><span style="color: gainsboro;">(</span></div><div> name=fn<span style="color: gainsboro;">,</span> length=<span style="color: #dcdcaa;">len</span><span style="color: gainsboro;">(</span>uploaded<span style="color: gainsboro;">[</span>fn<span style="color: gainsboro;">])))</span></div><div> </div><div><span style="color: #6aa94f;"># Then move kaggle.json into the folder where the API expects to find it.</span></div><div><span style="color: #82c6ff;">!</span>mkdir -p ~/<span style="color: gainsboro;">.</span>kaggle/ && mv kaggle<span style="color: gainsboro;">.</span>json ~/<span style="color: gainsboro;">.</span>kaggle/ && chmod <span style="color: #b5cea8;">600</span> ~/<span style="color: gainsboro;">.</span>kaggle/kaggle<span style="color: gainsboro;">.</span>json</div><div><br /></div><div><div style="line-height: 19px;"><div><span style="color: #82c6ff;">!</span>pip install keras_nlp --upgrade</div><div><span style="color: #82c6ff;">!</span>pip install keras --upgrade</div><br /><div><span style="color: #c586c0;">import</span> os</div><br /><div>os.environ<span style="color: gainsboro;">[</span><span style="color: #ce9178;">"KERAS_BACKEND"</span><span style="color: gainsboro;">]</span> = <span style="color: #ce9178;">"jax"</span></div><br /><div><span style="color: #c586c0;">import</span> keras_nlp</div><div><span style="color: #c586c0;">import</span> keras</div><div><span style="color: #c586c0;">import</span> tensorflow <span style="color: #c586c0;">as</span> tf</div><div><span style="color: #c586c0;">import</span> time</div><br /><div>keras.mixed_precision.set_global_policy<span style="color: gainsboro;">(</span><span style="color: #ce9178;">"mixed_float16"</span><span style="color: gainsboro;">)</span></div><div><span style="color: gainsboro;"><br /></span></div><div><div style="line-height: 19px;"><div>preprocessor = keras_nlp.models.GemmaPreprocessor.from_preset<span style="color: gainsboro;">(</span></div><div> <span style="color: #ce9178;">"gemma_2b_en"</span></div><div><span style="color: gainsboro;">)</span></div><br /><div>gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset<span style="color: gainsboro;">(</span><span style="color: #ce9178;">"gemma_2b_en"</span><span style="color: gainsboro;">)</span></div><div><span style="color: gainsboro;"><br /></span></div><div><div style="line-height: 19px;"><div style="line-height: 19px;">gemma_lm.generate<span style="color: gainsboro;">(</span><span style="color: #ce9178;">"which one came first, egg or chicken?"</span><span style="color: gainsboro;">,</span> max_length=<span style="color: #b5cea8;">130</span><span style="color: gainsboro;">)</span></div></div></div></div></div></div></div><div><br /></div></div></div></div><p> </p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-67797277816102582192024-02-01T02:20:00.000-08:002024-02-01T02:22:50.880-08:00Step by Step Guide to Configure Amazon Bedrock with VPC Endpoints and PrivateLink<p>This video is step by step tutorial to setup AWS Bedrock with VPC Endpoints and PrivateLink to build secure and private generative AI applications.</p><span><a name='more'></a></span><p></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="353" src="https://www.youtube.com/embed/ybWH1K5yLIw" width="479" youtube-src-id="ybWH1K5yLIw"></iframe></div><br /><b><br /></b><p></p><p><b> Steps:</b></p><p><br /></p><p>Step 0: Make sure that Private subnet with private route table without any route to internet is there.</p><p>Step 1: Create 2 SG = Bedrock-Endpoint-SG and Bedrock-Lambda-SG</p><p>Step 2: In Bedrock-Lambda-SG , ADD Bedrock-EndPoint-SG for all traffic in INBOUND, and OUTBOUND FOR 0.0.0.0</p><p>Step 3: In Bedrock-EndpointP-SG, Add Bedrock-Lambda-SG for all traffic in INBOUND and OUTBOUND </p><p>Step 4: Create 2 endpoints bedrock, bedrock-runtime in private subnet and attach Bedrock-EndpointP-SG with both</p><p>Step 5: Create lambda function, set time to 15 seconds, and attach Bedrock-Lambda-SG, lambda execution role should have bedrock permissions</p><p><br /></p><p><b>Lambda Code:</b></p><p><br /></p><p>import boto3</p><p>import json</p><p><br /></p><p>def lambda_handler(event,context):</p><p> bedrock = boto3.client(</p><p> service_name='bedrock', </p><p> region_name='us-east-1'</p><p> )</p><p> </p><p> # Bedrock Runtime client used to invoke and question the models</p><p> bedrock_runtime = boto3.client(</p><p> service_name='bedrock-runtime', </p><p> region_name='us-east-1'</p><p> )</p><p><br /></p><p> models = bedrock.list_foundation_models().get('modelSummaries')</p><p><br /></p><p> for model in models:</p><p> print(model['modelName'] + ', Input=' + '-'.join(model['inputModalities']) + ', Output=' + ''.join(model['outputModalities']) + ', Provider=' + model['providerName'])</p><p> </p><p> return{</p><p> <span style="white-space: pre;"> </span>'statusCode':200,</p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>}</span></p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-74225829125377096452024-01-25T22:17:00.000-08:002024-01-25T22:17:51.526-08:00How to Identify Oracle Database Orphan Sessions<p> In the world of database management, particularly with Oracle databases, "orphan sessions" are a common issue that can affect performance and resource utilization. </p><span><a name='more'></a></span><p>In Oracle databases, an orphan session, sometimes known as a "zombie session," is a session that remains in the database even though its corresponding client process has terminated. These sessions no longer have a user actively interacting with them, yet they consume system resources and can hold locks, leading to performance degradation and blocking issues.</p><p>Orphan sessions can occur due to various reasons such as:</p><p></p><ul style="text-align: left;"><li>Network issues that disrupt the connection between the client and the server.</li><li>Application or client crashes that terminate the session abnormally.</li><li>Database bugs or misconfigurations.</li></ul><div><br /></div><div><b>Queries to Identify Orphan Sessions:</b></div><div><br /></div><p></p><div>SELECT s.sid, s.serial#, p.spid, s.username, s.program</div><div><div>FROM v$session s</div><div>JOIN v$process p ON p.addr = s.paddr</div><div>WHERE s.type != 'BACKGROUND';</div></div><div><br /></div><div>This query lists active sessions, excluding background processes. It provides session identifiers (sid, serial#), the operating system process identifier (spid), and the username and program name. Orphan sessions often show NULL or unusual entries in the program column.</div><div><br /></div><div><div>SELECT s.sid, s.serial#, p.spid, s.username, s.program</div><div>FROM v$session s</div><div>JOIN v$process p ON p.addr = s.paddr</div><div>WHERE s.type != 'BACKGROUND'</div><div>AND NOT EXISTS (SELECT NULL FROM v$process WHERE spid = s.process);</div></div><div><br /></div><div>This query filters the sessions where the client process (spid) associated with the session does not exist in the v$process view, indicating a potential orphan.</div><div><br /></div><div><br /></div><div><div>SELECT s.sid, s.serial#, l.object_id, o.object_name, o.object_type</div><div>FROM v$session s</div><div>JOIN dba_objects o ON o.object_id = l.object_id</div><div>JOIN v$lock l ON s.sid = l.sid</div><div>WHERE s.sid IN (SELECT sid FROM v$session WHERE ... /* Conditions from above queries */);</div></div><div><br /></div><div><div><br /></div><div>This query identifies locks held by sessions suspected to be orphans, which is useful for understanding the impact of these sessions on the database.</div></div><div><br /></div><div><b>How to Manage Orphan Sessions:</b></div><div><br /></div><div><div>Manual Termination: Using the ALTER SYSTEM KILL SESSION command to terminate the identified orphan sessions. Or Kill at OS level with kill -9 spid command.</div><div><br /></div><div>Automated Monitoring and Cleanup: Implementing automated scripts or database jobs to periodically identify and clean up orphan sessions.</div><div><br /></div><div>Prevention: Addressing the root causes, such as network stability and application robustness, can reduce the occurrence of orphan sessions.</div></div><div><br /></div>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-75731846150493459072024-01-25T21:47:00.000-08:002024-01-25T21:47:24.266-08:00Oracle OCI's Generative AI Service: A New Era in Cloud Computing<p> The world of cloud computing is witnessing a revolutionary change with the introduction of Oracle Cloud Infrastructure's (OCI) Generative AI Service. This innovative offering from Oracle is a testament to the rapidly evolving field of artificial intelligence (AI), particularly in the realm of generative models. As businesses and developers seek more efficient and creative solutions, Oracle's new service stands out as a significant milestone.</p><p><br /></p><p>What is Oracle OCI's Generative AI Service?</p><p>Oracle's OCI Generative AI Service is a cloud-based platform that provides users with access to powerful generative AI models. These models are capable of creating a wide range of content, including text, images, and possibly even audio or video in the future. The service is designed to integrate seamlessly with other OCI offerings, ensuring a cohesive and efficient cloud computing experience.</p><p><br /></p><p>Key Features and Capabilities</p><p>Advanced AI Models</p><p>At the heart of OCI's Generative AI Service are state-of-the-art AI models that have been trained on vast datasets. These models can generate high-quality, original content based on user inputs, making them invaluable for a variety of applications.</p><p><br /></p><p>Scalability and Performance</p><p>Oracle's robust cloud infrastructure ensures that the Generative AI Service can scale to meet the demands of any project, big or small. This scalability is crucial for handling large-scale AI tasks without compromising on performance or speed.</p><p><br /></p><p>Integration with OCI Ecosystem</p><p>The service is designed to work seamlessly with other OCI products, such as data storage, analytics, and security services. This integration allows for a more streamlined workflow, as users can easily access and combine different OCI services.</p><p><br /></p><p>Use Cases</p><p>The potential applications of Oracle OCI's Generative AI Service are vast and varied. Here are a few examples:</p><p><br /></p><p>Content Creation</p><p>For marketers and content creators, the service can generate written content, images, and potentially other forms of media. This capability can significantly speed up the content creation process and inspire new ideas.</p><p><br /></p><p>Business Intelligence</p><p>Businesses can leverage the AI's ability to analyze and synthesize information to gain insights from data. This can aid in decision-making, trend analysis, and strategy development.</p><span><a name='more'></a></span><p>Research and Development</p><p>In the R&D sector, the service can assist in generating hypotheses, modeling complex systems, and even predicting outcomes, thereby accelerating the pace of innovation.</p><p><br /></p><p>Security and Ethics</p><p>Oracle recognizes the importance of ethical AI use and has implemented measures to ensure the responsible deployment of its Generative AI Service. This includes safeguards against generating harmful or biased content and maintaining user privacy and data security.</p><p><br /></p><p>Getting Started with OCI Generative AI Service</p><p>To start using the service, users need to have an Oracle Cloud account. Oracle provides comprehensive documentation and support to help users integrate the AI service into their projects.</p><p><br /></p><p>Conclusion</p><p>Oracle OCI's Generative AI Service is a groundbreaking addition to the cloud computing landscape. It offers immense potential for businesses, developers, and creators to harness the power of AI for generating content and gaining insights. As the technology continues to evolve, it will be exciting to see the innovative applications that emerge from this platform.</p><p><br /></p><p>Oracle's commitment to integrating advanced AI capabilities into its cloud services is a clear indicator of the transformative impact AI is set to have across industries. The OCI Generative AI Service is not just a tool; it's a gateway to a future where AI and cloud computing work hand in hand to unlock new possibilities.</p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-53823187760119338402024-01-17T23:27:00.000-08:002024-01-17T23:27:51.791-08:00Top Code LLM in the World - Locally Install Stable Code 3B without GPU <p>This video walks through step by step guide to locally install top code AI Model which can run on CPU and its very small in size. </p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="398" src="https://www.youtube.com/embed/iVb6yHGWjS0" width="479" youtube-src-id="iVb6yHGWjS0"></iframe></div><br /><p><br /></p><p><b>Code:</b></p><p><br /></p><pre style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #f9fafb; border-color: rgb(229, 231, 235); border-radius: 0.375rem; border-style: solid; border-width: 0px; box-sizing: border-box; font-feature-settings: normal; font-variation-settings: normal; line-height: 1.71429; margin-bottom: 1.71429em; margin-top: 1.71429em; overflow-x: auto; padding: 0.857143em 1.14286em;"><code class="language-python" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-text-opacity: 1; --tw-translate-x: 0; --tw-translate-y: 0; background-color: transparent; border-color: rgb(229, 231, 235); border-radius: 0px; border-style: solid; border-width: 0px; box-sizing: border-box; font-feature-settings: normal; font-variation-settings: normal; line-height: 1.25rem; padding: 0px;"><span class="hljs-keyword" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a626a4; font-family: inherit; font-size: 0.875rem;">pip install transformers torch</span></code></pre><pre style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #f9fafb; border-color: rgb(229, 231, 235); border-radius: 0.375rem; border-style: solid; border-width: 0px; box-sizing: border-box; font-feature-settings: normal; font-variation-settings: normal; line-height: 1.71429; margin-bottom: 1.71429em; margin-top: 1.71429em; overflow-x: auto; padding: 0.857143em 1.14286em;"><code class="language-python" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-text-opacity: 1; --tw-translate-x: 0; --tw-translate-y: 0; background-color: transparent; border-color: rgb(229, 231, 235); border-radius: 0px; border-style: solid; border-width: 0px; box-sizing: border-box; font-feature-settings: normal; font-variation-settings: normal; line-height: 1.25rem; padding: 0px;"><span class="hljs-keyword" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a626a4; font-family: inherit; font-size: 0.875rem;">import</span><span style="color: rgba(0, 0, 0, 0); font-family: inherit;"><span style="font-size: 0.875rem;"> torch
</span></span><span class="hljs-keyword" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a626a4; font-family: inherit; font-size: 0.875rem;">from</span><span style="color: rgba(0, 0, 0, 0); font-family: inherit;"><span style="font-size: 0.875rem;"> transformers </span></span><span class="hljs-keyword" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a626a4; font-family: inherit; font-size: 0.875rem;">import</span><span style="color: rgba(0, 0, 0, 0); font-family: inherit;"><span style="font-size: 0.875rem;"> AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained(</span></span><span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f; font-family: inherit; font-size: 0.875rem;">"stabilityai/stable-code-3b"</span><span style="color: rgba(0, 0, 0, 0); font-family: inherit;"><span style="font-size: 0.875rem;">, trust_remote_code=</span></span><span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb; font-family: inherit; font-size: 0.875rem;">True</span><span style="color: rgba(0, 0, 0, 0); font-family: inherit;"><span style="font-size: 0.875rem;">)
model = AutoModelForCausalLM.from_pretrained(
</span></span><span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f; font-family: inherit; font-size: 0.875rem;">"stabilityai/stable-code-3b"</span><span style="color: rgba(0, 0, 0, 0); font-family: inherit;"><span style="font-size: 0.875rem;">,
trust_remote_code=</span></span><span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb; font-family: inherit; font-size: 0.875rem;">True</span><span style="color: rgba(0, 0, 0, 0); font-family: inherit;"><span style="font-size: 0.875rem;">,
torch_dtype=</span></span><span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f; font-family: inherit; font-size: 0.875rem;">"auto"</span><span style="color: rgba(0, 0, 0, 0); font-family: inherit;"><span style="font-size: 0.875rem;">,
)
model.cuda()
</span></span><span style="color: #1f2937; font-family: IBM Plex Mono, ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, Liberation Mono, Courier New, monospace;"><span style="font-size: 14px;">inputs = tokenizer("write me a script in Java to reverse a list", return_tensors="pt").to(model.device)
tokens = model.generate(
**inputs,
max_new_tokens=500,
temperature=0.2,
do_sample=True,
)
print(tokenizer.decode(tokens[0], skip_special_tokens=True))</span></span></code></pre>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-46584744359955495292024-01-14T16:40:00.000-08:002024-01-14T16:40:25.424-08:00How to Install NVIDIA Drivers on AWS EC2 Instance Windows<p> This video shows how to install NVIDIA drivers for Windows in AWS EC2 Instance G4DN and other instance types.</p><span><a name='more'></a></span><p><br /></p><p></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="405" src="https://www.youtube.com/embed/rsTtxx7kZyQ" width="487" youtube-src-id="rsTtxx7kZyQ"></iframe></div><br /><b><br /></b><p></p><p><b>Commands Used:</b></p><p><br /></p><p>msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi</p><p><br /></p><p>aws --version </p><p><br /></p><p>In new windows, aws configure and set your IAM user key id and secret access key</p><p><br /></p><p>Run below in Powershell as administrator:</p><p><br /></p><p>Install-Module -Name AWS.Tools.Installer</p><p><br /></p><p>$Bucket = "ec2-windows-nvidia-drivers"</p><p>$KeyPrefix = "latest"</p><p>$LocalPath = "$home\Desktop\NVIDIA"</p><p>$Objects = Get-S3Object -BucketName $Bucket -KeyPrefix $KeyPrefix -Region us-east-1</p><p>foreach ($Object in $Objects) {</p><p> $LocalFileName = $Object.Key</p><p> if ($LocalFileName -ne '' -and $Object.Size -ne 0) {</p><p> $LocalFilePath = Join-Path $LocalPath $LocalFileName</p><p> Copy-S3Object -BucketName $Bucket -Key $Object.Key -LocalFile $LocalFilePath -Region us-east-1</p><p> }</p><p>}</p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-15057216172217631902024-01-13T21:26:00.000-08:002024-01-13T21:26:23.754-08:00Talk with Comics Using AI in Any Language<p> This video shows step by step demo with code as how to analyze comics in any language and talk to them using LlamaIndex and ChatGPT.</p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="365" src="https://www.youtube.com/embed/1Ih_JUvADYk" width="439" youtube-src-id="1Ih_JUvADYk"></iframe></div><br /><p><br /></p><p><br /></p><p><b>Code Used:</b></p><div style="background-color: #1e1e1e; color: #d4d4d4; font-family: monospace, Consolas, "Courier New", monospace; font-size: 14px; line-height: 19px; white-space: pre;"><div><span style="color: #82c6ff;">%pip </span>install llama_index ftfy regex tqdm</div><div><span style="color: #82c6ff;">%pip </span>install git+https<span style="color: gainsboro;">:</span>//github.com/openai/CLIP.git</div><div><span style="color: #82c6ff;">%pip </span>install torch torchvision</div><div><span style="color: #82c6ff;">%pip </span>install matplotlib scikit-image</div><div><span style="color: #82c6ff;">%pip </span>install -U qdrant_client</div><div><br /></div><div><div style="line-height: 19px;"><div><span style="color: #c586c0;">import</span> os</div><br /><div>openai_api_key = os.environ<span style="color: gainsboro;">[</span><span style="color: #ce9178;">'OPENAI_API_KEY'</span><span style="color: gainsboro;">]</span></div><div><span style="color: gainsboro;"><br /></span></div><div><div style="line-height: 19px;"><div><span style="color: #c586c0;">from</span> PIL <span style="color: #c586c0;">import</span> Image</div><div><span style="color: #c586c0;">import</span> matplotlib.pyplot <span style="color: #c586c0;">as</span> plt</div><div><span style="color: #c586c0;">import</span> os</div><br /><div>image_paths = <span style="color: gainsboro;">[]</span></div><div><span style="color: #c586c0;">for</span> img_path <span style="color: #82c6ff;">in</span> os.listdir<span style="color: gainsboro;">(</span><span style="color: #ce9178;">"./urdu"</span><span style="color: gainsboro;">):</span></div><div> image_paths.append<span style="color: gainsboro;">(</span><span style="color: #4ec9b0;">str</span><span style="color: gainsboro;">(</span>os.path.join<span style="color: gainsboro;">(</span><span style="color: #ce9178;">"./urdu"</span><span style="color: gainsboro;">,</span> img_path<span style="color: gainsboro;">)))</span></div><br /><br /><div><span style="color: #569cd6;">def</span> <span style="color: #dcdcaa;">plot_images</span>(<span style="color: #9cdcfe;">image_paths</span>)<span style="color: gainsboro;">:</span></div><div> images_shown = <span style="color: #b5cea8;">0</span></div><div> plt.figure<span style="color: gainsboro;">(</span>figsize=<span style="color: gainsboro;">(</span><span style="color: #b5cea8;">25</span><span style="color: gainsboro;">,</span> <span style="color: #b5cea8;">12</span><span style="color: gainsboro;">))</span></div><div> <span style="color: #c586c0;">for</span> img_path <span style="color: #82c6ff;">in</span> image_paths<span style="color: gainsboro;">:</span></div><div> <span style="color: #c586c0;">if</span> os.path.isfile<span style="color: gainsboro;">(</span>img_path<span style="color: gainsboro;">):</span></div><div> image = Image.<span style="color: #dcdcaa;">open</span><span style="color: gainsboro;">(</span>img_path<span style="color: gainsboro;">)</span></div><br /><div> plt.subplot<span style="color: gainsboro;">(</span><span style="color: #b5cea8;">2</span><span style="color: gainsboro;">,</span> <span style="color: #b5cea8;">2</span><span style="color: gainsboro;">,</span> images_shown + <span style="color: #b5cea8;">1</span><span style="color: gainsboro;">)</span></div><div> plt.imshow<span style="color: gainsboro;">(</span>image<span style="color: gainsboro;">)</span></div><div> plt.xticks<span style="color: gainsboro;">([])</span></div><div> plt.yticks<span style="color: gainsboro;">([])</span></div><br /><div> images_shown += <span style="color: #b5cea8;">1</span></div><div> <span style="color: #c586c0;">if</span> images_shown >= <span style="color: #b5cea8;">9</span><span style="color: gainsboro;">:</span></div><div> <span style="color: #c586c0;">break</span></div><br /><br /><div>plot_images<span style="color: gainsboro;">(</span>image_paths<span style="color: gainsboro;">)</span></div><div><span style="color: gainsboro;"><br /></span></div><div><span style="color: gainsboro;"><br /></span></div><div><div style="line-height: 19px;"><div><span style="color: #c586c0;">from</span> llama_index.multi_modal_llms.openai <span style="color: #c586c0;">import</span> OpenAIMultiModal</div><div><span style="color: #c586c0;">from</span> llama_index <span style="color: #c586c0;">import</span> SimpleDirectoryReader</div><br /><div>image_documents = SimpleDirectoryReader<span style="color: gainsboro;">(</span><span style="color: #ce9178;">"./urdu"</span><span style="color: gainsboro;">)</span>.load_data<span style="color: gainsboro;">()</span></div><br /><div>openai_mm_llm = OpenAIMultiModal<span style="color: gainsboro;">(</span></div><div> model=<span style="color: #ce9178;">"gpt-4-vision-preview"</span><span style="color: gainsboro;">,</span> api_key=openai_api_key<span style="color: gainsboro;">,</span> max_new_tokens=<span style="color: #b5cea8;">1500</span></div><div><span style="color: gainsboro;">)</span></div><br /><div>response_eng = openai_mm_llm.complete<span style="color: gainsboro;">(</span></div><div> prompt=<span style="color: #ce9178;">"Describe the comic strip panels as an alternative text"</span><span style="color: gainsboro;">,</span></div><div> image_documents=image_documents<span style="color: gainsboro;">,</span></div><div><span style="color: gainsboro;">)</span></div><br /><div><span style="color: #dcdcaa;">print</span><span style="color: gainsboro;">(</span>response_eng<span style="color: gainsboro;">)</span></div><div><span style="color: gainsboro;"><br /></span></div></div></div></div></div></div></div><div><br /></div></div>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-62374530534310851682024-01-08T21:26:00.000-08:002024-01-08T21:36:31.117-08:00Use AI to Query AWS RDS Database with LlamaIndex<p> This video shows step by step guide with code as how to integrate LlamaIndex with AWS RDS Postgresql database to query in natural language. Its AI and LLM at its best.</p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="374" src="https://www.youtube.com/embed/ki4TNlZNn5U" width="450" youtube-src-id="ki4TNlZNn5U"></iframe></div><br /><p><br /></p><p><br /></p><p>Commands Used:</p><p><br /></p><p><br /></p><p>sudo apt-get install libpq-dev</p><p>pip install llama-index sqlalchemy psycopg2</p><p><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">from</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> sqlalchemy </span><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">import</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> create_engine, MetaData</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">from</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> llama_index </span><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">import</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> SQLDatabase, VectorStoreIndex</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">from</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> llama_index.indices.struct_store </span><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">import</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> SQLTableRetrieverQueryEngine</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">from</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> llama_index.objects </span><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">import</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> SQLTableNodeMapping, ObjectIndex, SQLTableSchema</span></p><p>pg_uri = f"postgresql+psycopg2://postgres:test1234@<RDS Endpoint>:5432/testdb"</p><p><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">engine = create_engine(pg_uri)</span></p><p><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">metadata_obj = MetaData() </span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">metadata_obj.reflect(engine) </span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">sql_database = SQLDatabase(engine)</span></p><p><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">from</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> llama_index.indices.struct_store </span><span class="hljs-keyword" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #aa0d91; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">import</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> NLSQLTableQueryEngine</span></p><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">query_engine = NLSQLTableQueryEngine(</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> sql_database=sql_database,</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"> tables=[</span><span class="hljs-string" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #c41a16; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">"companies","contacts"</span><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">],</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">)</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;"><br /></span><div><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">query_str = </span><span class="hljs-string" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #c41a16; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">"who works in AWS?"</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><p><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">response = query_engine.query(query_str)</span></p><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">query_str = </span><span class="hljs-string" face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; box-sizing: inherit; color: #c41a16; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">"How many people work in GCP and what are there names?"</span><br style="background-color: #f9f9f9; box-sizing: inherit; color: #242424; font-family: source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace; font-size: 14px; letter-spacing: -0.308px; white-space: pre;" /><p><span face="source-code-pro, Menlo, Monaco, "Courier New", Courier, monospace" style="background-color: #f9f9f9; color: #242424; font-size: 14px; letter-spacing: -0.308px; white-space: pre;">response = query_engine.query(query_str)</span></p><p> </p><p><br /></p></div>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-31716565052568310732024-01-05T10:11:00.000-08:002024-01-05T10:11:02.811-08:00Train TinyLlama 1.1B Locally on Own Custom Dataset<p> This video explains in easy and simple tutorial as how to train or fine-tune TinyLlama model locally by using unsloth on your own data.</p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="335" src="https://www.youtube.com/embed/VoDHpnCN6PA" width="480" youtube-src-id="VoDHpnCN6PA"></iframe></div><br /><p><b>Code Used:</b></p><p><br /></p><p>import torch</p><p>major_version, minor_version = torch.cuda.get_device_capability()</p><p><br /></p><p>!pip install "unsloth[colab] @ git+https://github.com/unslothai/unsloth.git"</p><p><br /></p><p>from unsloth import FastLanguageModel</p><p>import torch</p><p>max_seq_length = 4096</p><p>dtype = None</p><p>load_in_4bit = True</p><p><br /></p><p>model, tokenizer = FastLanguageModel.from_pretrained(</p><p> model_name = "unsloth/tinyllama-bnb-4bit",</p><p> max_seq_length = max_seq_length,</p><p> dtype = dtype,</p><p> load_in_4bit = load_in_4bit,</p><p>)</p><p><br /></p><p>model = FastLanguageModel.get_peft_model(</p><p> model,</p><p> r = 32, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128</p><p> target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",</p><p> "gate_proj", "up_proj", "down_proj",],</p><p> lora_alpha = 32,</p><p> lora_dropout = 0,</p><p> bias = "none", </p><p> use_gradient_checkpointing = False,</p><p> random_state = 3407,</p><p> max_seq_length = max_seq_length,</p><p>)</p><p><br /></p><p>from trl import SFTTrainer</p><p>from transformers import TrainingArguments</p><p>from transformers.utils import logging</p><p>logging.set_verbosity_info()</p><p><br /></p><p>trainer = SFTTrainer(</p><p> model = model,</p><p> train_dataset = dataset,</p><p> dataset_text_field = "text",</p><p> max_seq_length = max_seq_length,</p><p> packing = True, </p><p> args = TrainingArguments(</p><p> per_device_train_batch_size = 2,</p><p> gradient_accumulation_steps = 4,</p><p> warmup_ratio = 0.1,</p><p> num_train_epochs = 1,</p><p> learning_rate = 2e-5,</p><p> fp16 = not torch.cuda.is_bf16_supported(),</p><p> bf16 = torch.cuda.is_bf16_supported(),</p><p> logging_steps = 1,</p><p> optim = "adamw_8bit",</p><p> weight_decay = 0.1,</p><p> lr_scheduler_type = "linear",</p><p> seed = 3407,</p><p> output_dir = "outputs",</p><p> ),</p><p>)</p><p><br /></p><p>trainer_stats = trainer.train()</p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-73254270956863803422023-12-13T16:26:00.000-08:002023-12-13T16:26:39.324-08:00How to Build RAG Pipeline with Mixtral 8x7B to Talk to Your Own Documents<p> This video shows step by step process as how to locally build RAG pipeline with Mixtral 8x7B to talk to local documents in PDF etc.</p><span><a name='more'></a></span><p><b><br /></b></p><p></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="404" src="https://www.youtube.com/embed/_CBBz2lCR5U" width="487" youtube-src-id="_CBBz2lCR5U"></iframe></div><br /><b><br /></b><p></p><p><b><br /></b></p><p><b>Commands Used:</b></p><p><br /></p><p>#%%capture</p><p><br /></p><p>!pip install farm-haystack[colab]</p><p><br /></p><p>from getpass import getpass</p><p>HF_TOKEN = getpass("Hugging Face Token")</p><p><br /></p><p>from haystack.nodes import PreProcessor,PromptModel, PromptTemplate, PromptNode</p><p><br /></p><p>from google.colab import files</p><p>files.upload()</p><p><br /></p><p>%%capture</p><p>!pip install PyPDF2</p><p><br /></p><p>import PyPDF2</p><p>from haystack import Document</p><p><br /></p><p>pdf_file_path = "e10897.pdf" # Sostituisci con il percorso del tuo file PDF</p><p><br /></p><p>def extract_text_from_pdf(pdf_path):</p><p> text = ""</p><p> with open(pdf_path, "rb") as pdf_file:</p><p> pdf_reader = PyPDF2.PdfReader(pdf_file)</p><p> for page_num in range(len(pdf_reader.pages)):</p><p> page = pdf_reader.pages[page_num]</p><p> text += page.extract_text()</p><p><br /></p><p> return text</p><p><br /></p><p>pdf_text = extract_text_from_pdf(pdf_file_path)</p><p><br /></p><p># Creazione del documento di Haystack</p><p>doc = Document(</p><p> content=pdf_text,</p><p> meta={"pdf_path": pdf_file_path}</p><p>)</p><p><br /></p><p>docs = [doc]</p><p><br /></p><p>processor = PreProcessor(</p><p> clean_empty_lines=True,</p><p> clean_whitespace=True,</p><p> clean_header_footer=True,</p><p> split_by="word",</p><p> split_length=500,</p><p> split_respect_sentence_boundary=True,</p><p> split_overlap=0,</p><p> language="it",</p><p>)</p><p><br /></p><p>preprocessed_docs = processor.process(docs)</p><p><br /></p><p>from haystack.document_stores import InMemoryDocumentStore</p><p><br /></p><p>document_store = InMemoryDocumentStore(use_bm25=True)</p><p>document_store.write_documents(preprocessed_docs)</p><p><br /></p><p>from haystack import Pipeline</p><p>from haystack.nodes import BM25Retriever</p><p>retriever = BM25Retriever(document_store, top_k=2)</p><p><br /></p><p>qa_template = PromptTemplate(prompt=</p><p> """ Using only the information contained in the context,</p><p> answer only the question asked without adding suggestions of possible questions and answer exclusively in Italian.</p><p> If the answer cannot be deduced from the context, reply: "\I don't know because it is not relevant to the Context.\"</p><p> Context: {join(documents)};</p><p> Question: {query}</p><p> """)</p><p><br /></p><p>prompt_node = PromptNode(</p><p> model_name_or_path="mistralai/Mixtral-8x7B-Instruct-v0.1",</p><p> api_key=HF_TOKEN,</p><p> default_prompt_template=qa_template,</p><p> max_length=500,</p><p> model_kwargs={"model_max_length": 5000}</p><p>)</p><p><br /></p><p>rag_pipeline = Pipeline()</p><p>rag_pipeline.add_node(component=retriever, name="retriever", inputs=["Query"])</p><p>rag_pipeline.add_node(component=prompt_node, name="prompt_node", inputs=["retriever"])</p><p><br /></p><p>from pprint import pprint</p><p>print_answer = lambda out: pprint(out["results"][0].strip())</p><p><br /></p><p>print_answer(rag_pipeline.run(query="What is Oracle DBA?"))</p><p><br /></p><p>print_answer(rag_pipeline.run(query="Why Lion is king of jungle?"))</p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-86735348661112664542023-12-11T20:39:00.000-08:002023-12-12T20:13:16.376-08:00 Mixtral 8X7B Local Installation - Step by StepThis is simple tutorial to locally install Mixtral 8*7B. <span><a name='more'></a></span><div><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="337" src="https://www.youtube.com/embed/ExoN9GnkzPs" width="476" youtube-src-id="ExoN9GnkzPs"></iframe></div><br /><div><br /></div><div><br /></div><div><pre style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #f9fafb; border-color: rgb(229, 231, 235); border-radius: 0.375rem; border-style: solid; border-width: 0px; box-sizing: border-box; color: #4b5563; font-family: "IBM Plex Mono", ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace; font-size: 0.875em; line-height: 1.71429; margin-bottom: 1.71429em; margin-top: 1.71429em; overflow-x: auto; padding: 0.857143em 1.14286em;"><span color="rgb(31 41 55 / var(--tw-text-opacity))" style="background-color: transparent; font-family: inherit; font-size: 0.875rem;">pip3 install --upgrade transformers optimum</span></pre><pre style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #f9fafb; border-color: rgb(229, 231, 235); border-radius: 0.375rem; border-style: solid; border-width: 0px; box-sizing: border-box; color: #4b5563; font-family: "IBM Plex Mono", ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace; font-size: 0.875em; line-height: 1.71429; margin-bottom: 1.71429em; margin-top: 1.71429em; overflow-x: auto; padding: 0.857143em 1.14286em;"><code class="language-shell" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-text-opacity: 1; --tw-translate-x: 0; --tw-translate-y: 0; background-color: transparent; border-color: rgb(229, 231, 235); border-radius: 0px; border-style: solid; border-width: 0px; box-sizing: border-box; color: rgb(31 41 55 / var(--tw-text-opacity)); font-family: inherit; font-size: 0.875rem; line-height: 1.25rem; padding: 0px;">pip3 uninstall -y auto-gptq
git clone https://github.com/PanQiWei/AutoGPTQ
cd AutoGPTQ
git checkout v0.5.1
pip3 install .</code></pre><pre style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; background-color: #f9fafb; border-color: rgb(229, 231, 235); border-radius: 0.375rem; border-style: solid; border-width: 0px; box-sizing: border-box; color: #4b5563; font-family: "IBM Plex Mono", ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace; font-size: 0.875em; line-height: 1.71429; margin-bottom: 1.71429em; margin-top: 1.71429em; overflow-x: auto; padding: 0.857143em 1.14286em;"><code class="language-python" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-text-opacity: 1; --tw-translate-x: 0; --tw-translate-y: 0; background-color: transparent; border-color: rgb(229, 231, 235); border-radius: 0px; border-style: solid; border-width: 0px; box-sizing: border-box; color: rgb(31 41 55 / var(--tw-text-opacity)); font-family: inherit; font-size: 0.875rem; line-height: 1.25rem; padding: 0px;">model_name_or_path = <span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f;">"TheBloke/Mixtral-8x7B-Instruct-v0.1-GPTQ"</span>
<span class="hljs-keyword" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a626a4;">from</span> transformers <span class="hljs-keyword" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a626a4;">import</span> AutoModelForCausalLM, AutoTokenizer, pipeline, GPTQConfig
<span class="hljs-keyword" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a626a4;">from</span> auto_gptq <span class="hljs-keyword" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a626a4;">import</span> AutoGPTQForCausalLM
model_name_or_path = args.model_dir
<span class="hljs-comment" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a0a1a7; font-style: italic;"># To use a different branch, change revision</span>
<span class="hljs-comment" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #a0a1a7; font-style: italic;"># For example: revision="gptq-4bit-32g-actorder_True"</span>
model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
model_basename=<span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f;">"model"</span>,
use_safetensors=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">True</span>,
trust_remote_code=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">False</span>,
device=<span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f;">"cuda:0"</span>,
use_triton=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">False</span>,
disable_exllama=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">False</span>,
disable_exllamav2=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">True</span>,
quantize_config=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">None</span>)
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">True</span>, trust_remote_code=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">False</span>)
prompt = <span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f;">"Why Lion is King of Jungle?"</span>
prompt_template=<span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f;">f'''<s>[INST] <span class="hljs-subst" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #e45649;">{prompt}</span> [/INST]</span>
<span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f;">'''</span>
<span class="hljs-built_in" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #c18401;">print</span>(<span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f;">"\n\n*** Generate:"</span>)
input_ids = tokenizer(prompt_template, return_tensors=<span class="hljs-string" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #50a14f;">'pt'</span>).input_ids.cuda()
output = model.generate(inputs=input_ids, temperature=<span class="hljs-number" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #986801;">0.7</span>, do_sample=<span class="hljs-literal" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #0184bb;">True</span>, top_p=<span class="hljs-number" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #986801;">0.95</span>, top_k=<span class="hljs-number" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #986801;">40</span>, max_new_tokens=<span class="hljs-number" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #986801;">512</span>)
<span class="hljs-built_in" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #c18401;">print</span>(tokenizer.decode(output[<span class="hljs-number" style="--tw-border-spacing-x: 0; --tw-border-spacing-y: 0; --tw-ring-color: rgb(59 130 246 / .5); --tw-ring-offset-color: #fff; --tw-ring-offset-shadow: 0 0 #0000; --tw-ring-offset-width: 0px; --tw-ring-shadow: 0 0 #0000; --tw-rotate: 0; --tw-scale-x: 1; --tw-scale-y: 1; --tw-scroll-snap-strictness: proximity; --tw-shadow-colored: 0 0 #0000; --tw-shadow: 0 0 #0000; --tw-skew-x: 0; --tw-skew-y: 0; --tw-translate-x: 0; --tw-translate-y: 0; border-color: rgb(229, 231, 235); border-style: solid; border-width: 0px; box-sizing: border-box; color: #986801;">0</span>]))</code></pre></div>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-20397186524738399942023-11-16T14:02:00.000-08:002023-11-16T14:02:18.364-08:00AWS PartyRock - Amazon Bedrock AI Playground<p>With PartyRock, you can build AI apps in seconds for free by using latest LLMs and without any code very easily.</p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="342" src="https://www.youtube.com/embed/jlMUVYiYBh8" width="482" youtube-src-id="jlMUVYiYBh8"></iframe></div><br /><p><br /></p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-84343301869169624092023-10-25T23:29:00.008-07:002023-10-25T23:37:00.259-07:00Beginner Tutorial to Fine-Tune an AI Model<p> This video steps through an easy tutorial to fine-tune a model on custom dataset from scratch by using LlamaIndex and Gradient.</p><span><a name='more'></a></span><p><br /></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="325" src="https://www.youtube.com/embed/-sTKPumMirE" width="460" youtube-src-id="-sTKPumMirE"></iframe></div><br /><p><br /></p><p><b>Dataset Used:</b></p><p><br /></p><p>{"inputs": "<s>### Instruction:\nWho is Fahd Mirza?\n\n### Response:\nFahd Mirza is an AI Cloud Engineer based in Sydney Australia. He has also got a background in databases and devops plus infrastrucutre.</s>"}</p><p>{"inputs": "<s>### Instruction:\nWhat are hobbies of Fahd Mirza?\n\n### Response\nFahd Mirza loves to spend time on his youtube channel and reading about technology.</s>"}</p><p>{"inputs": "<s>### Instruction:\nWhat Fahd Mirza's favorite Color?\n\n### Response:\nFahd Mirza's favorite color varies from time to time. These days its blue.</s>"}</p><p>{"inputs": "<s>### Instruction:\nWhat does Fahd Mirza look like?\n\n### Response:\nFahd Mirza looks like a human.</s>"}</p><p><br /></p><p><b>.env File:</b></p><p><br /></p><p>GRADIENT_ACCESS_TOKEN='<>'</p><p>GRADIENT_WORKSPACE_ID='<>'</p><p><br /></p><p><b>Commands Used:</b></p><p><br /></p><p>!pip install llama-index gradientai -q</p><p>!pip install python-dotenv </p><p><br /></p><p>import os</p><p>from dotenv import load_dotenv, find_dotenv</p><p>_= load_dotenv(find_dotenv())</p><p><br /></p><p>questions = [</p><p> "Who is Fahd Mirza??",</p><p> "What is Fahd Mirza's favorite Color?",</p><p> "What are hobbies of Fahd Mirza?",</p><p>]</p><p><br /></p><p>prompts = list(</p><p> f"<s> ### Instruction:\n{q}\n\n###Response:\n" for q in questions</p><p>)</p><p><br /></p><p>print(prompts)</p><p><br /></p><p>import os</p><p>from llama_index.llms import GradientBaseModelLLM</p><p>from llama_index.finetuning.gradient.base import GradientFinetuneEngine</p><p><br /></p><p>base_model_slug = "nous-hermes2"</p><p>base_model_llm = GradientBaseModelLLM(</p><p> base_model_slug=base_model_slug, max_tokens=100</p><p>)</p><p><br /></p><p>base_model_responses = list(base_model_llm.complete(p).text for p in prompts)</p><p><br /></p><p>finetune_engine = GradientFinetuneEngine(</p><p> base_model_slug=base_model_slug,</p><p> name="my test finetune engine model adapter",</p><p> data_path="data.jsonl",</p><p>)</p><p><br /></p><p>epochs = 2</p><p>for i in range(epochs):</p><p> finetune_engine.finetune()</p><p>fine_tuned_model = finetune_engine.get_finetuned_model(max_tokens=100)</p><p><br /></p><p>fine_tuned_model_responses = list(</p><p> fine_tuned_model.complete(p).text for p in prompts</p><p>)</p><p>fine_tuned_model._model.delete()</p><p><br /></p><p>for i, q in enumerate(questions):</p><p> print(f"Question: {q}")</p><p> print(f"Base: {base_model_responses[i]}")</p><p> print(f"Fine tuned: {fine_tuned_model_responses[i]}")</p><p> print()</p><p><span style="white-space: pre;"> </span></p><p><span style="white-space: pre;"> </span></p><p><br /></p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-68411131033300831442023-10-25T21:29:00.004-07:002023-10-25T21:29:33.680-07:00Setting Environment Variable in Google Colab<p>This video shows how to set environment variable and load them in Google Colab notebook, AWS Sagemaker notebook or Jupyter notebook.</p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="405" src="https://www.youtube.com/embed/t_pFWwLtza0" width="488" youtube-src-id="t_pFWwLtza0"></iframe></div><div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><b>Commands Used:</b></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div class="separator" style="clear: both; text-align: left;"><br /></div><div>import os</div><div>from dotenv import load_dotenv, find_dotenv</div><div>_= load_dotenv(find_dotenv())</div><p><br /></p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-51848045442495925142023-10-19T20:25:00.004-07:002023-10-19T20:25:42.647-07:00Step by Step Mistral 7B Installation Local on Linux Windows or in Cloud<p> This is detailed tutorial as how to locally install Mistral 7B model in AWS, Linux, Windows, or anywhere you like.</p><span><a name='more'></a></span><p><br /></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="379" src="https://www.youtube.com/embed/2knnWtL9090" width="456" youtube-src-id="2knnWtL9090"></iframe></div><br /><p><br /></p><p><br /></p><p>Commands Used:</p><p><br /></p><p>pip3 install optimum</p><p>pip3 install git+https://github.com/huggingface/transformers.git@72958fcd3c98a7afdc61f953aa58c544ebda2f79</p><p><br /></p><p>git clone https://github.com/PanQiWei/AutoGPTQ</p><p>cd AutoGPTQ</p><p>git checkout v0.4.2</p><p>pip3 install .</p><p><br /></p><p><br /></p><p>from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline</p><p><br /></p><p>model_name_or_path = "TheBloke/SlimOpenOrca-Mistral-7B-GPTQ"</p><p># To use a different branch, change revision</p><p># For example: revision="gptq-4bit-32g-actorder_True"</p><p><br /></p><p>model = AutoModelForCausalLM.from_pretrained(model_name_or_path,</p><p> device_map="auto",</p><p> trust_remote_code=False,</p><p> revision="main")</p><p><br /></p><p>tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)</p><p><br /></p><p>system_message = "You are an expert at bathroom renovations."</p><p>prompt = """</p><p>Renovate the following old bathroom:</p><p>I have a 25 year old house with an old bathroom. I want to renovate it completely. </p><p>Think about it step by step, and give me steps to renovate the bathroom. Also give me cost of every step in Australian dollars.</p><p>"""</p><p><br /></p><p>prompt_template=f'''<|im_start|>system</p><p>{system_message}<|im_end|></p><p><|im_start|>user</p><p>{prompt}<|im_end|></p><p><|im_start|>assistant</p><p>'''</p><p><br /></p><p>print("\n\n*** Generate:")</p><p><br /></p><p>input_ids = tokenizer(prompt_template, return_tensors='pt').input_ids.cuda()</p><p>output = model.generate(inputs=input_ids, temperature=0.7, do_sample=True, top_p=0.95, top_k=40, max_new_tokens=512)</p><p>print(tokenizer.decode(output[0]))</p><p><br /></p><p># Inference can also be done using transformers' pipeline</p><p><br /></p><p>print("*** Pipeline:")</p><p>pipe = pipeline(</p><p> "text-generation",</p><p> model=model,</p><p> tokenizer=tokenizer,</p><p> max_new_tokens=512,</p><p> do_sample=True,</p><p> temperature=0.7,</p><p> top_p=0.95,</p><p> top_k=40,</p><p> repetition_penalty=1.1</p><p>)</p><p><br /></p><p>print(pipe(prompt_template)[0]['generated_text'])</p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com1tag:blogger.com,1999:blog-3496259157130184660.post-50891893020608715732023-10-15T21:18:00.002-07:002023-10-15T21:23:35.851-07:00Step by Step Fine-Tuning Mistral 7B with Custom Dataset<p>Large Language Models are trained on huge amount of data. Falcon 40B model, e.g. has been trained on 1 trillion tokens with 40 billion parameters. This training took around 2 months and 384 GPUs on AWS. </p><span><a name='more'></a></span><p><br /></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="382" src="https://www.youtube.com/embed/rVczNN6PbW0" width="489" youtube-src-id="rVczNN6PbW0"></iframe></div><br /><p><br /></p><p>If you want to use these LLMs for your own data, then you need to adapt them or fine-tune them. Fine-tuning a model larger than 10B is an expensive and time consuming task. </p><p><br /></p><p>This is where HuggingFace's PEFT library comes handy. PEFT stands for parameter efficent fine tuning. We can use a fine-tuning technique called as QLORA to train LLMs on our own dataset in far less time using far less resources. QLORA stands for Quantized Low Rank Adapation and allows us to to train a small portion of model without losing much efficieny. After the training is completed, there is no necessity to save the entire model, as the base model remains frozen.</p><p><br /></p><p>Python Package Installation:</p><p>===================================</p><p>We begin by installing all the required dependencies. </p><p><br /></p><p>- The Huggingface Transformer Reinforcement Learning (TRL) library simplifies Reinforcement Learning from Human Feedback (RLHF) settings. </p><p>- Transformers is a Python library that makes downloading and training state-of-the-art ML models easy.</p><p>- Accelerate is a library that enables the same PyTorch code to be run across any distributed configuration by adding just four lines of code</p><p>- Parameter-Efficient Fine-Tuning (PEFT) methods enable efficient adaptation of pre-trained language models (PLMs) to various downstream applications without fine-tuning all the model's parameters.</p><p>- Datasets is a library for easily accessing and sharing datasets for Audio, Computer Vision, and Natural Language Processing (NLP) tasks. </p><p>- Bitsandbytes is a lightweight wrapper around CUDA custom functions, in particular 8-bit optimizers and quantization functions.</p><p>- einops stands for Einstein-Inspired Notation for operations. It is an open-source python framework for writing deep learning code in a new and better way.</p><p>- Tiktoken is an open-source tool developed by OpenAI that is utilized for tokenizing text. Tokenization is when you split a text string to a list of tokens. Tokens can be letters, words or grouping of words</p><p>- By using wandb, you can track, compare, explain and reproduce machine learning experiments.</p><p>- xFormers is a PyTorch based library which hosts flexible Transformers parts.</p><p>- SentencePiece is an unsupervised text tokenizer and detokenizer mainly for Neural Network-based text generation systems where the vocabulary size is predetermined prior to the neural model training.</p><p><br /></p><p><br /></p><p><br /></p><p>!pip intall -q trl transformers accelerate peft datasets bitsandbytes einops tiktoken wandb xformers sentencepiece</p><p><br /></p><p>Prepare Dataset:</p><p>===================================</p><p>I will be using Gath_baize dataset comprising approximately 210k prompts to train Mistral-7b. The dataset consists of a mixture of data from Alpaca, Stack Overflow, medical, and Quora datasets. In this load_dataset function we are loading the dataset with full train split as we are going to use this dataset in training. If we would be just testing it, then we would use split=test. </p><p><br /></p><p>from datasets import load_dataset</p><p>gathbaize = load_dataset("gathnex/Gath_baize",split="train")</p><p><br /></p><p>gathbaize</p><p><br /></p><p>gathbaize["chat_sample"][0]</p><p><br /></p><p>gathbaize_sampled = gathbaize.shuffle(seed=42).select(range(50))</p><p><br /></p><p>gathbaize_sampled["chat_sample"][0]</p><p><br /></p><p>Check for GPU:</p><p>===================================</p><p><br /></p><p>The NVIDIA System Management Interface (nvidia-smi) is a command line utility, based on top of the NVIDIA Management Library (NVML), intended to aid in the management and monitoring of NVIDIA GPU devices.</p><p><br /></p><p>!nvidia-smi</p><p><br /></p><p><br /></p><p>Create LLM Model:</p><p>===================================</p><p><br /></p><p>-Torch is an open source ML library used for creating deep neural networks </p><p>-AutoModelForCausalLM used for auto-regressive models. regressive means referring to previous state. Auto-regressive models predict future values based on past values.</p><p>-A tokenizer is responsible for preprocessing text into an array of numbers as inputs to a model.</p><p>-Bitsandbytes library simplifies the process of model quantization, making it more accessible and user-friendly.</p><p><br /></p><p>import torch</p><p>from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig</p><p>from peft import prepare_model_for_kbit_training => this prepares the model for fine-tuning.</p><p><br /></p><p>model_name = "ybelkada/Mistral-7B-v0.1-bf16-sharded"</p><p><br /></p><p><br /></p><p>- BitsandBytesConfig is configuration for QLORA. QLoRA reduces the memory usage of LLM finetuning without performance tradeoffs compared to standard 16-bit model finetuning. QLoRA uses 4-bit quantization to compress a pretrained language model. The LM parameters are then frozen and a relatively small number of trainable parameters are added to the model in the form of Low-Rank Adapters. During finetuning, QLoRA backpropagates gradients through the frozen 4-bit quantized pretrained language model into the Low-Rank Adapters. The LoRA layers are the only parameters being updated during training. </p><p><br /></p><p><br /></p><p>- The basic way to load a model in 4bit is to pass the argument load_in_4bit=True</p><p>- There are different variants of 4bit quantization such as NF4 (normalized float 4 (default)) or pure FP4 quantization. NF4 is better for performance.</p><p>- You can change the compute dtype of the quantized model by just changing the bnb_4bit_compute_dtype argument. A dtype (data type) object describes how the bytes in the fixed-size block of memory corresponding to an array item should be interpreted.</p><p>- bnb_4bit_use_double_quant uses a second quantization after the first one to save an additional 0.4 bits per parameter. </p><p><br /></p><p>bnb_config = BitsAndBytesConfig(</p><p> load_in_4bit= True,</p><p> bnb_4bit_quant_type= "nf4",</p><p> bnb_4bit_compute_dtype= torch.bfloat16,</p><p> bnb_4bit_use_double_quant= False,</p><p>)</p><p><br /></p><p>- Whether or not to allow for custom models defined on the Hub in their own modeling files. </p><p><br /></p><p>model = AutoModelForCausalLM.from_pretrained(</p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>model_name,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>quantization_config=bnb_config,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>trust_remote_code=True,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>device_map='auto'</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>)</span></p><p><br /></p><p>- When fine-tuning the model, you want to use the updated model params. Using the old (cached) values kinda defeats the purpose of finetuning. Hence, the past (cached) key values are disregarded for the fine-tuned model.</p><p>- Setting config.pretraining_tp to a value different than 1 will activate the more accurate but slower computation of the linear layers</p><p>- Gradient check-pointing is only needed if training leads to out-of-memory (OOM) errors so its a sort of best practice.</p><p><br /></p><p>model.config.use_cache = False</p><p>model.config.pretraining_tp = 1</p><p>model.gradient_checkpointing_enable()</p><p><br /></p><p>model = prepare_model_for_kbit_training(model)<span style="white-space: pre;"> </span></p><p><br /></p><p><br /></p><p>Create LLM Tokenizer:</p><p>===================================</p><p><br /></p><p>- Pad_token is a special token used to make arrays of tokens the same size for batching purpose.</p><p>- eos_token is a special token used as an end of sentence token</p><p>- bos_token is a special token representing the beginning of a sentence.</p><p><br /></p><p>tokenizer = AutoTokenizer.from_pretrained(model_name,trust_remote_code=True)</p><p>tokenizer.pad_token = tokenizer.eos_token</p><p><br /></p><p>tokenizer.add_eos_token = True</p><p>tokenizer.add_bos_token, tokenizer.add_eos_token</p><p><br /></p><p>from peft import LoraConfig, TaskType</p><p><br /></p><p>- LoraConfig allows you to control how LoRA is applied to the base model through the following parameters:</p><p><br /></p><p>lora_alpha: LoRA scaling factor.</p><p>r: the rank of the update matrices, expressed in int. Lower rank results in smaller update matrices with fewer trainable parameters.</p><p>bias: Specifies if the bias parameters should be trained. Can be 'none', 'all' or 'lora_only'.</p><p>target_modules: The modules (for example, attention blocks) to apply the LoRA update matrices.</p><p>(lora_dropout): This is the probability that each neuron's output is set to zero during training, used to prevent overfitting.</p><p><br /></p><p>peft_config = LoraConfig(</p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>lora_alpha=16,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>lora_dropout=0.05,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>r=16,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>bias="none",</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>task_type="CAUSAL_LM",</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>target_modules=["q_proj", "k_proj", "v_proj", "o_proj","gate_proj"]</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>)</span></p><p><span style="white-space: pre;"> </span></p><p>from peft import get_peft_model</p><p>model = get_peft_model(model,peft_config)<span style="white-space: pre;"> </span></p><p><br /></p><p>from transformers import TrainingArguments</p><p><br /></p><p>- num_train_epochs(`float`, *optional*, defaults to 3.0): Total number of training epochs to perform</p><p>- per_device_train_batch_size is the batch size per GPU/TPU core/CPU for training. </p><p>- Gradient accumulation is a technique that simulates a larger batch size by accumulating gradients from multiple small batches before performing a weight update. This technique can be helpful in scenarios where the available memory is limited, and the batch size that can fit in memory is small.</p><p>- This parameter tells the optimizer how far to move the weights in the direction opposite of the gradient for a mini-batch.</p><p>- warmup_ration is ratio of total training steps used for a linear warmup from 0 to learning_rate.</p><p>- max steps If set to a positive number, the total number of training steps to perform.</p><p><br /></p><p>training_arguments = TrainingArguments(</p><p> output_dir= "./results",</p><p> num_train_epochs= 1,</p><p> per_device_train_batch_size= 8,</p><p> gradient_accumulation_steps= 2,</p><p> optim = "paged_adamw_8bit",</p><p> save_steps= 5000,</p><p> logging_steps= 30,</p><p> learning_rate= 2e-4,</p><p> weight_decay= 0.001,</p><p> fp16= False,</p><p> bf16= False,</p><p> max_grad_norm= 0.3,</p><p> max_steps= -1,</p><p> warmup_ratio= 0.3,</p><p> group_by_length= True,</p><p> lr_scheduler_type= "constant"</p><p>)</p><p><br /></p><p>from trl import SFTTrainer</p><p><br /></p><p>- The SFTTrainer is a light wrapper around the transformers Trainer to easily fine-tune language models or adapters on a custom dataset.</p><p>- max_seq_length: maximum sequence length to use for the `ConstantLengthDataset` and for automaticallty creating the Dataset. Defaults to `512`.</p><p>- SFTTrainer supports example packing, where multiple short examples are packed in the same input sequence to increase training efficiency.</p><p> </p><p>trainer = SFTTrainer(</p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>model=model,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>train_dataset=gathbaize,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>peft_config=peft_config,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>dataset_text_field="chat_sample",</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>tokenizer=tokenizer,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>args=training_arguments,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>max_seq_length=None,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>packing= False,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>)</span></p><p><br /></p><p>trainer.train()</p><p><br /></p><p>Saving the Model:</p><p>=================</p><p><br /></p><p>trained_model_dir = './trained_model'<span style="white-space: pre;"> </span></p><p>model.save_pretrained(trained_model_dir)</p><p><br /></p><p>Load the Trained Model:</p><p>======================</p><p><br /></p><p>from peft import PeftConfig, PeftModel</p><p>config = PeftConfig.from_pretrained(trained_model_dir)</p><p><br /></p><p>trained_model = AutoModelForCausalLM.from_pretrained(</p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>config.base_model_name_or_path,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>return_dict=True,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>quantization_config=bnb_config,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>trust_remote_code=True,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>device_map='auto'</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>)</span></p><p><span style="white-space: pre;"> </span></p><p>trained_model = PeftModel.from_pretrained(trained_model,trained_model_dir)</p><p><br /></p><p>trained_model_tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path,trust_remote_code=True)<span style="white-space: pre;"> </span></p><p>trained_model_tokenizer.pad_token = trained_model_tokenizer.eos_token</p><p><br /></p><p><br /></p><p>Create Generation Config for Prediction:</p><p>=======================================</p><p><br /></p><p>generation_config = trained_model.generation_config</p><p>generation_config.max_new_token = 1024</p><p>generation_config.tempreture = 0.7</p><p>generation_config.top_p = 0.7</p><p>generation_config.num_return_sequence = 1</p><p>generation_config.pad_token_id = trained_model_tokenizer.pad_token_id</p><p>generation_config.eos_token_id = trained_model_tokenizer.eos_token_id</p><p><br /></p><p>generation_config</p><p><br /></p><p>Model Inference:</p><p>================</p><p><br /></p><p>device = 'cuda:0'</p><p><br /></p><p>query = 'larget text to be summarized'</p><p><br /></p><p><br /></p><p>user_prompt = 'Explain large language models'</p><p>system_prompt = 'The conversation between Human and AI assisatance named MyMistral\n'</p><p>B_INST, E_INST = "[INST]", "[/INST]"</p><p>prompt = f"{system_prompt}{B_INST}{user_prompt.strip()}\n{E_INST}"</p><p><br /></p><p><br /></p><p><br /></p><p>encodings = trained_model_tokenizer(prompt, return_tensors='pt').to(device)</p><p><br /></p><p>encodings </p><p><br /></p><p>with torch.inference_mode():</p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>outputs = trained_model.generate(</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>input_ids=encodings.input_ids,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>attention_mask=encodings.attention_mask,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>generation_config=generation_config,</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>max_new_token=100</span></p><p><span style="white-space: normal;"><span style="white-space: pre;"> </span>)</span></p><p><br /></p><p>outputs </p><p><br /></p><p>outputs = trained_model_tokenizer.decode(outputs[0],skip_special_tokens=True)</p><p><br /></p><p>outputs </p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-78717437306710375172023-10-12T17:13:00.001-07:002023-10-12T17:25:06.955-07:00AlloyDB Omni with Vertex AI Installation Locally in AWS<p> This video is step by step tutorial to install AlloyDB Omni with Vertex AI support locally in AWS.</p><span><a name='more'></a></span><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="347" src="https://www.youtube.com/embed/VMw6jr9JOHU" width="477" youtube-src-id="VMw6jr9JOHU"></iframe></div><br /><p><br /></p><p><b>Commands Used:</b></p><p><br /></p><p> 2 sudo curl https://get.docker.com | sh && sudo systemctl --now enable docker</p><p> 3 sudo apt-get update</p><p> 4 sudo groupadd docker</p><p> 5 sudo usermod -aG docker ${USER}</p><p> 6 sudo systemctl restart docker</p><p> 7 stat -fc %T /sys/fs/cgroup/</p><p> 8 sudo apt-get install apt-transport-https ca-certificates gnupg curl sudo</p><p> 9 echo "deb [signed-by=/usr/share/keyrings/cloud.google.asc] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list</p><p> 10 curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -</p><p> 11 sudo apt-get update && sudo apt-get install google-cloud-cli</p><p> 12 gcloud init</p><p> 13 cat /etc/*release</p><p> 14 curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo tee /usr/share/keyrings/cloud.google.asc</p><p> 15 sudo apt-get update && sudo apt-get install google-cloud-cli</p><p> 16 gcloud init</p><p> 17 curl https://asia-apt.pkg.dev/doc/repo-signing-key.gpg | sudo apt-key add -</p><p> 18 sudo apt update</p><p> 19 echo "deb https://asia-apt.pkg.dev/projects/alloydb-omni alloydb-omni-apt main" | sudo tee -a /etc/apt/sources.list.d/artifact-registry.list</p><p> 20 sudo apt update</p><p> 21 sudo apt-get install alloydb-cli</p><p> 22 sudo alloydb system-check</p><p> 23 df -hT</p><p> 24 cd /</p><p> 25 ls</p><p> 29 sudo mkdir /alloydb</p><p> 32 sudo chown ubuntu:ubuntu /alloydb</p><p> 33 sudo chmod 777 /alloydb</p><p> 49 sudo alloydb database-server install --data-dir=/alloydb --enable-alloydb-ai=true --private-key-file-path=/home/ubuntu/key.json --vertex-ai-region="us-central1"</p><p> 50 sudo alloydb database-server start</p><p> 56 docker exec -it pg-service psql -h localhost -U postgres</p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-76801088306964951962023-10-06T15:30:00.005-07:002023-10-12T17:25:54.045-07:00Tutorial Amazon Bedrock to Create Chatbot with Persona<p> This video tutorial shows how code and step by step description with demo as how to use AWS Bedrock to create chatbot with persona.</p><span><a name='more'></a></span><p></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="387" src="https://www.youtube.com/embed/4lBbG0Dxbts" width="465" youtube-src-id="4lBbG0Dxbts"></iframe></div><br />Code:<p></p><div style="background-color: #1f1f1f; color: #cccccc; font-family: Consolas, "Courier New", monospace; font-size: 16px; line-height: 22px; white-space: pre;"><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">boto3</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">json</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">os</span></div><div><span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">sys</span></div><br /><div><span style="color: #c586c0;">from</span> <span style="color: #4ec9b0;">langchain</span>.<span style="color: #4ec9b0;">chains</span> <span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">ConversationChain</span></div><div><span style="color: #c586c0;">from</span> <span style="color: #4ec9b0;">langchain</span>.<span style="color: #4ec9b0;">llms</span>.<span style="color: #4ec9b0;">bedrock</span> <span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">Bedrock</span></div><div><span style="color: #c586c0;">from</span> <span style="color: #4ec9b0;">langchain</span>.<span style="color: #4ec9b0;">memory</span> <span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">ConversationBufferMemory</span></div><br /><div><span style="color: #c586c0;">from</span> <span style="color: #4ec9b0;">langchain</span>.<span style="color: #4ec9b0;">prompts</span> <span style="color: #c586c0;">import</span> <span style="color: #4ec9b0;">PromptTemplate</span></div><br /><div><span style="color: #9cdcfe;">template</span> <span style="color: #d4d4d4;">=</span> <span style="color: #ce9178;">"""The following is a friendly conversation between a human and an AI. </span></div><div><span style="color: #ce9178;"> The AI is talkative and provides lots of specific details from its context. </span></div><div><span style="color: #ce9178;"> If the AI does not know the answer to a question, it truthfully says it does not know.</span></div><div><span style="color: #ce9178;">Current conversation:</span></div><div><span style="color: #569cd6;">{history}</span></div><div><span style="color: #ce9178;">Human: </span><span style="color: #569cd6;">{input}</span></div><div><span style="color: #ce9178;">Assistant:"""</span></div><br /><div><span style="color: #9cdcfe;">claude_prompt</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">PromptTemplate</span>(<span style="color: #9cdcfe;">input_variables</span><span style="color: #d4d4d4;">=</span>[<span style="color: #ce9178;">"history"</span>, <span style="color: #ce9178;">"input"</span>], <span style="color: #9cdcfe;">template</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">template</span>)</div><br /><br /><div><span style="color: #9cdcfe;">bedrock</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">boto3</span>.<span style="color: #dcdcaa;">client</span>(</div><div> <span style="color: #9cdcfe;">service_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'bedrock-runtime'</span>, </div><div> <span style="color: #9cdcfe;">region_name</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">'us-east-1'</span></div><div>)</div><br /><div><span style="color: #9cdcfe;">memory</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">ConversationBufferMemory</span>(<span style="color: #9cdcfe;">ai_prefix</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">"Assistant"</span>)</div><div><span style="color: #9cdcfe;">memory</span>.<span style="color: #9cdcfe;">chat_memory</span>.<span style="color: #dcdcaa;">add_user_message</span>(<span style="color: #ce9178;">"You will be acting as a Plumber but you might also give answers to non-plumbing questions."</span>)</div><div><span style="color: #9cdcfe;">memory</span>.<span style="color: #9cdcfe;">chat_memory</span>.<span style="color: #dcdcaa;">add_ai_message</span>(<span style="color: #ce9178;">"I am a Plumber and give professional answers"</span>)</div><br /><div><span style="color: #9cdcfe;">cl_llm</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">Bedrock</span>(<span style="color: #9cdcfe;">model_id</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">"anthropic.claude-v2"</span>,<span style="color: #9cdcfe;">client</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">bedrock</span>)</div><br /><div><span style="color: #9cdcfe;">conversation</span> <span style="color: #d4d4d4;">=</span> <span style="color: #4ec9b0;">ConversationChain</span>(</div><div> <span style="color: #9cdcfe;">llm</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">cl_llm</span>, <span style="color: #9cdcfe;">verbose</span><span style="color: #d4d4d4;">=</span><span style="color: #569cd6;">True</span>, <span style="color: #9cdcfe;">memory</span><span style="color: #d4d4d4;">=</span><span style="color: #9cdcfe;">memory</span></div><div>)</div><br /><div><span style="color: #9cdcfe;">conversation</span>.<span style="color: #9cdcfe;">prompt</span> <span style="color: #d4d4d4;">=</span> <span style="color: #9cdcfe;">claude_prompt</span></div><br /><div><span style="color: #6a9955;">#print(conversation.predict(input="What are steps to renovate a bathroom?"))</span></div><div><span style="color: #6a9955;">#print(conversation.predict(input="How do you fix a leaking tap?"))</span></div><div><span style="color: #dcdcaa;">print</span>(<span style="color: #9cdcfe;">conversation</span>.<span style="color: #dcdcaa;">predict</span>(<span style="color: #9cdcfe;">input</span><span style="color: #d4d4d4;">=</span><span style="color: #ce9178;">"how to write a python program to reverse a list?"</span>))</div><br /><br /></div>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0tag:blogger.com,1999:blog-3496259157130184660.post-35893412785208587872023-09-16T19:52:00.005-07:002023-09-16T19:52:23.490-07:00Clone Any Voice with AI - Locally Install XTTS Model<p> <span style="background-color: rgba(0, 0, 0, 0.05); color: #0f0f0f; font-family: Roboto, Arial, sans-serif; font-size: 14px; white-space: pre-wrap;">This video shows in step by step tutorial as how to install and run Coqui XTTS model locally. TTS is a Voice generation model that lets you clone voices into different languages by using just a quick 3-second audio clip.</span></p><p><span></span></p><a name='more'></a><span style="background-color: rgba(0, 0, 0, 0.05); color: #0f0f0f; font-family: Roboto, Arial, sans-serif; font-size: 14px; white-space: pre-wrap;"><br /></span><p></p><p></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="357" src="https://www.youtube.com/embed/1eZx_2phFVI" width="488" youtube-src-id="1eZx_2phFVI"></iframe></div><br /><span style="background-color: rgba(0, 0, 0, 0.05); color: #0f0f0f; font-family: Roboto, Arial, sans-serif; font-size: 14px; white-space: pre-wrap;"><br /></span><p></p><p><span style="background-color: rgba(0, 0, 0, 0.05); color: #0f0f0f; font-family: Roboto, Arial, sans-serif; font-size: 14px; white-space: pre-wrap;">Commands Used:</span></p><p><span style="background-color: rgba(0, 0, 0, 0.05); color: #0f0f0f; font-family: Roboto, Arial, sans-serif; font-size: 14px; white-space: pre-wrap;"><br /></span></p><p><span style="background-color: rgba(0, 0, 0, 0.05); font-size: 14px; white-space: pre-wrap;"><span style="color: #0f0f0f; font-family: Roboto, Arial, sans-serif;">!pip install transformers
!pip install tts
from TTS.api import TTS
tts = TTS("tts_models/multilingual/multi-dataset/xtts_v1", gpu=True)
tts.tts_to_file(text="This is my new cloned voice in AI. If you like, don't forget to subscribe to this channel.",
file_path="output.wav",
speaker_wav="speaker.wav",
language="en")</span></span></p>Fahd Mirzahttp://www.blogger.com/profile/14722451950835849728noreply@blogger.com0