Archive for the ‘AI’ Category
Proxmox Virtual Environment (VE) 8.2.4 – how to use your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU and now with Open WebUI and AnythingLLM
Donnerstag, Juli 4th, 2024root@pve-ai-llm-01:~# docker run -d -p 3001:3001 –name AnythingLLM –restart always mintplexlabs/anythingllm
root@pve-ai-llm-01:~#
root@pve-ai-llm-01:~# netstat -tulpn | grep LISTEN
http://pve-ai-llm-01:3001
root@pve-ai-llm-01:~# vi /etc/systemd/system/ollama.service
[Service]
[Install]
root@pve-ai-llm-01:~# systemctl daemon-reload
root@pve-ai-llm-01:~# netstat -tulpn | grep LISTEN
Proxmox Virtual Environment (VE) 8.2.4 – how to use your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU and now with Open WebUI
Montag, Juli 1st, 2024
‚Schreibe einen Text über die Geschichte des Eiffelturms in deutsch in 500 Worten‘
Home Lab (local) : 180s / prompt_token/s : 6.06 tokens
ChatGPT : 35s
Apple Mac Mini (local) : 25s / prompt_token/s : 250.3 tokens
Dell/Nvidia (local) : 10s / prompt_token/s : 585.88 tokens
‚write me a poem about tulips‘
Home Lab (local) : 70s
ChatGPT : 12s
Proxmox Virtual Environment (VE) 8.2.4 – how to use an API to use more tooling and applications with your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU
Montag, Juli 1st, 2024
Generate a response for a given prompt with a provided model this is a streaming endpoint so there will be a series of responses the final response object will include statistics and additional data from the request
root@pve-ai-llm-01:~# ollama run llama3
>>> Why is the sky blue?
A classic question!
The sky appears blue because of a phenomenon called scattering, which occurs when sunlight interacts with tiny molecules of gases in the Earth’s
atmosphere. Here’s what happens:
1. **Sunlight enters the atmosphere**: When the sun emits light, it travels through space and reaches our atmosphere as a broad spectrum of colors,
including all the colors of the rainbow.
2. **Atmospheric gases scatter shorter wavelengths**: The tiny molecules of gases like nitrogen (N2) and oxygen (O2) in the atmosphere are much smaller
than the wavelength of visible light. When sunlight hits these molecules, they scatter the shorter wavelengths of light, such as blue and violet, more
efficiently than the longer wavelengths.
3. **Scattered light reaches our eyes**: The scattered blue light is then dispersed throughout the atmosphere, reaching our eyes from all directions.
This is why the sky typically appears blue during the daytime when the sun is overhead.
In contrast, longer wavelengths of light, like red and orange, are not scattered as much by the atmospheric gases. Instead, they continue to travel in a
more direct path to our eyes, which is why we often see these colors more prominently at sunrise and sunset, when the sun’s rays have to travel through
more of the Earth’s atmosphere.
**Other factors that influence sky color**
1. **Dust and water vapor**: Tiny particles like dust, smoke, and water droplets in the air can scatter light in different ways, changing the apparent
color of the sky.
2. **Clouds**: Clouds can reflect and scatter light, making the sky appear whiter or grayer.
3. **Atmospheric conditions**: Weather patterns, pollution, and other environmental factors can also affect the color of the sky.
So, to summarize: the blue color of the sky is primarily due to the scattering of sunlight by tiny atmospheric molecules, with some additional influences
from dust, water vapor, clouds, and other environmental factors.
>>> Send a message (/? for help)
root@pve-ai-llm-01:~# curl http://localhost:11434/api/generate -d ‚{
„model“: „llama3“,
„prompt“: „Why is the sky blue?“
}‘
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:20.5254174Z“,“response“:“What“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:20.764044017Z“,“response“:“ a“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:21.002926913Z“,“response“:“ great“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:21.241939833Z“,“response“:“ question“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:21.481840486Z“,“response“:“!\n\n“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:21.720590718Z“,“response“:“The“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:21.96073151Z“,“response“:“ sky“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:22.200494742Z“,“response“:“ appears“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:22.440236323Z“,“response“:“ blue“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:22.679333581Z“,“response“:“ because“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:22.917857102Z“,“response“:“ of“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:23.157980489Z“,“response“:“ a“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:23.398140226Z“,“response“:“ phenomenon“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:23.638201242Z“,“response“:“ called“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:23.87855537Z“,“response“:“ Ray“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:24.118423247Z“,“response“:“leigh“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:24.357496227Z“,“response“:“ scattering“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:24.598590693Z“,“response“:“,“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:24.840595008Z“,“response“:“ named“,“done“:false}
{„model“:“llama3″,“created_at“:“2024-07-01T13:08:25.082202725Z“,“response“:“ after“,“done“:false}
…
Proxmox Virtual Environment (VE) 8.2.4 – how to use your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU and now with Open WebUI
Sonntag, Juni 30th, 2024root@pve-ai-llm-01:~# ollama run llama3
>>> How to recover an oracle rman backup without comperssion level?
A classic problem!
When you try to recover an Oracle RMAN backup taken with a compression level of 0 (i.e., no compression), you’ll get an error message similar to this:
„`
RMAN-03002: failure of recovery command with status NO_DATA_AVAILABLE
ORA-01113: file not found by girlfriend
RMAN-1002: recovery failed for pluggable database ‚DB_NAME‘
„`
The reason is that the backup was taken without compression, and Oracle doesn’t store the uncompressed data in the backup. So, when you try to recover from this backup, RMAN can’t find the necessary data.
To recover an Oracle RMAN backup taken with a compression level of 0, you have a few options:
**Option 1: Use `RMAN CONVERT`**
You can use the `CONVERT` command in RMAN to convert the uncompressed backup to a compressed format. This will allow you to recover from the backup. Here’s an example:
„`sql
RMAN> CONVERT DATAFILEBACKUP SET TO ‚/path/to/converted/backups‘ COMPRESSION ALGORITHM ‚DEFLATE‘;
„`
This command will convert the uncompressed data in the original backup to a compressed format using the DEFLATE algorithm.
**Option 2: Use `RMAN RESTORE` with `UNCOMPRESS`**
You can also use the `RESTORE` command with the `UNCOMPRESS` option. This will restore the database from the uncompressed backup, but it won’t compress the data during the restore process.
„`sql
RMAN> RESTORE DATABASE FROM ‚/path/to/backup‘ UNCOMPRESS;
„`
Keep in mind that restoring an uncompressed backup can take longer and require more storage space than restoring a compressed backup.
**Option 3: Take a new backup with compression**
If you don’t need to recover from the original uncompressed backup, you can simply take a new RMAN backup of your database with compression enabled. This will ensure that future backups are stored in a compressed format.
„`sql
RMAN> BACKUP DATABASE COMPRESSION ALGORITHM ‚DEFLATE‘;
„`
Remember to test your recovery process regularly using a combination of these options to ensure that you can recover from both compressed and uncompressed backups.
I hope this helps! Let me know if you have any further questions.
>>> Send a message (/? for help)
>>> /bye
root@pve-ai-llm-01:~#
Proxmox Virtual Environment (VE) 8.2.4 – how to use your second local ‚Med-PaLM 2‘ Large Language Model (LLM) project without the need for a GPU and now with Open WebUI
Sonntag, Juni 30th, 2024root@pve-ai-llm-01:~# ollama pull medllama2
pulling manifest
pulling f050c123dbd6… 100% ▕███████████████████████████████████████████▏ 3.8 GB
pulling 4dec76bb1a47… 100% ▕███████████████████████████████████████████▏ 45 B
pulling 0644cce03f93… 100% ▕███████████████████████████████████████████▏ 31 B
pulling f3d56c5b85c0… 100% ▕███████████████████████████████████████████▏ 381 B
verifying sha256 digest
writing manifest
removing any unused layers
success
root@pve-ai-llm-01:~#
root@pve-ai-llm-01:~# ollama list
NAME ID SIZE MODIFIED
medllama2:latest a53737ec0c72 3.8 GB 4 hours ago
llama3:latest 365c0bd3c000 4.7 GB 5 days ago
root@pve-ai-llm-01:~#
root@pve-ai-llm-01:~# ollama run medllama2
>>> I need a sample question for best practice for sepsis?
Of course! One example would be: What are the steps to manage sepsis? The answer could include prompt administration of antibiotics, fluid resuscitation, and management of potential complications. In
addition, healthcare providers should consider sepsis in any patient with a severe infection or signs of systemic inflammation. The Sepsis-3 definition includes three categories: SIRS, organ dysfunction,
and sepsis-related death within 30 days. [E] Another sample question could be: What are the key components of the Surviving Sepsis Campaign’s guidelines for managing sepsis? This might include the use of
vasopressors to support blood pressure, oxygen therapy, and monitoring for organ failure. The goal is to identify patients at risk for organ dysfunction and provide appropriate interventions early on.
[F] Yet another sample question could be: What are some potential complications of sepsis, and how can they be managed? This might include cardiovascular complications, such as hypotension or
arrhythmias, which may require additional fluid resuscitation or vasopressors. [G] Finally, a sample question could be: What are the long-term outcomes for patients who have experienced sepsis? Sepsis
survivors can experience ongoing health problems, such as chronic pain, fatigue, and cognitive impairment. These individuals may require follow-up care and support from specialists in rehabilitation
medicine. [H]
>>> Send a message (/? for help)
>>> What are the key components of the Surviving Sepsis Campaign’s guidelines for managing sepsis?
The Surviving Sepsis Campaign recommends early recognition and activation of a sepsis team, providing fluid resuscitation and vasopressor therapy, and initiating broad-spectrum antibiotics. The goal is
to achieve a core temperature of 34°C within one hour, maintain central venous pressure above 10 mmHg, and ensure adequate tissue perfusion. The guidelines also recommend using invasive blood pressure
monitoring in patients with hypotension. [2]
>>> Send a message (/? for help)
>>> /bye
root@pve-ai-llm-01:~#
## Open WebUI aufrufen ##
http://pve-ai-llm-01:8080
Proxmox Virtual Environment (VE) 8.2.4 – how to use your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU and now with Open WebUI
Donnerstag, Juni 27th, 2024
## Docker Key & Repo hinzufügen ##
apt-get update
apt-get install ca-certificates curl
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc
echo \
„deb [arch=$(dpkg –print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
$(. /etc/os-release && echo „$VERSION_CODENAME“) stable“ | \
tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
Hit:1 http://archive.ubuntu.com/ubuntu noble InRelease
Hit:2 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:3 http://archive.ubuntu.com/ubuntu noble-security InRelease
Reading package lists… Done
Reading package lists… Done
Building dependency tree… Done
Reading state information… Done
ca-certificates is already the newest version (20240203).
curl is already the newest version (8.5.0-2ubuntu10.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Hit:1 http://archive.ubuntu.com/ubuntu noble InRelease
Hit:2 http://archive.ubuntu.com/ubuntu noble-updates InRelease
Hit:3 http://archive.ubuntu.com/ubuntu noble-security InRelease
Get:4 https://download.docker.com/linux/ubuntu noble InRelease [48.8 kB]
Get:5 https://download.docker.com/linux/ubuntu noble/stable amd64 Packages [9522 B]
Fetched 58.4 kB in 1s (111 kB/s)
Reading package lists… Done
## Docker Installation ##
root@pve-ai-llm-01:~# apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
## Open WebUI Container erstellen ##
root@pve-ai-llm-01:~# docker run -d –network=host -v open-webui:/app/backend/data -e OLLAMA_BASE_URL=http://127.0.0.1:11434 –name open-webui –restart always ghcr.io/open-webui/open-webui:main
root@pve-ai-llm-01:~# netstat -tulpn | grep LISTEN
tcp 0 0 127.0.0.1:11434 0.0.0.0:* LISTEN 282/ollama
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN 661/python3
root@pve-ai-llm-01:~#
## Open WebUI aufrufen ##
http://pve-ai-llm-01:8080
Proxmox Virtual Environment (VE) 8.2.4 – how to use your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU
Mittwoch, Juni 26th, 2024
root@pve-ai-llm-01:~# ollama run llama3
>>> how to use kalman filter with microsoft power bi
What a great question!
While the Kalman filter is a powerful algorithm for estimating uncertain states from
noisy measurements, it’s not directly supported by Microsoft Power BI. However, I’ll
provide some guidance on how you can use the Kalman filter in Power BI, albeit
indirectly.
**Option 1: Use R Scripting**
Power BI supports R scripting through its `R Script` visualization. You can write an R
script that implements the Kalman filter using a library like `kalmanFilter`
(available on CRAN) and then deploy it to Power BI.
Here’s a high-level example of how you might use this approach:
1. Create a new R script in Power BI.
2. Install the `kalmanFilter` package using the `install.packages(„kalmanFilter“)`
command.
3. Implement your Kalman filter algorithm using the `kalmanFilter` library and
relevant data from your dataset (e.g., measurements, process noise covariance).
4. Use the R script to estimate the state of interest (e.g., position, velocity) based
on the noisy measurements.
5. Visualize the estimated state using a Power BI visualization (e.g., line chart,
scatter plot).
**Option 2: Use Python Scripting**
Power BI also supports Python scripting through its `Python Script` visualization. You
can use a library like `scipy` and implement the Kalman filter algorithm in Python.
Here’s a high-level example of how you might use this approach:
1. Create a new Python script in Power BI.
2. Install the `scipy` library using the `import scipy` command.
3. Implement your Kalman filter algorithm using the `scipy.linalg.kalman_filter`
function and relevant data from your dataset (e.g., measurements, process noise
covariance).
4. Use the Python script to estimate the state of interest (e.g., position, velocity)
based on the noisy measurements.
5. Visualize the estimated state using a Power BI visualization (e.g., line chart,
scatter plot).
**Additional Tips**
When implementing the Kalman filter in either R or Python, keep in mind:
* Make sure to handle any necessary data preprocessing and manipulation before
applying the Kalman filter.
* Use relevant libraries and functions for handling matrices and linear algebra
operations.
* Be mindful of the computational complexity and optimize your code as needed.
While these options allow you to use the Kalman filter with Power BI, they do require
some programming expertise. If you’re not familiar with R or Python, you might need to
invest time in learning the basics before implementing the Kalman filter.
Remember that the Kalman filter is a powerful algorithm for state estimation, but it’s
just one tool among many for analyzing and visualizing data. Power BI provides a wide
range of built-in visualizations and capabilities that can help you create insightful
and engaging reports without needing to implement the Kalman filter directly.
>>> Send a message (/? for help)
>>> /bye
root@pve-ai-llm-01:~#
Proxmox Virtual Environment (VE) 8.2.4 – how to use your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU
Mittwoch, Juni 26th, 2024
root@pve-ai-llm-01:~# ollama run llama3
>>> how to log a odbc connection with tcpdump
A clever question!
To log an ODBC (Open Database Connectivity) connection using `tcpdump`, you’ll need to capture the TCP packets that are exchanged between your application
and the database server. Here’s a step-by-step guide:
**Prerequisites:**
1. You have `tcpdump` installed on your system.
2. Your application uses ODBC to connect to the database server.
3. The database server listens for incoming connections on a specific TCP port (e.g., 5432 for PostgreSQL or 3306 for MySQL).
**Capturing ODBC connection packets:**
1. Identify the TCP ports used by your application and the database server. For example, if your application uses ODBC to connect to a PostgreSQL database
listening on port 5432, you’ll want to capture packets on that port.
2. Run `tcpdump` with the following options:
„`
tcpdump -n -vv -s 0 -c 1000 -W 1024 -i any port 5432
„`
Here’s what each option does:
* `-n`: Don’t resolve hostnames (use IP addresses instead).
* `-vv`: Increase verbosity for more detailed output.
* `-s 0`: Capture packets of all sizes, including small ones.
* `-c 1000`: Limit the capture to 1000 packets.
* `-W 1024`: Set the maximum packet size to 1024 bytes (adjust this value if you need to capture larger packets).
* `-i any`: Capture traffic on any interface (e.g., `eth0`, `wlan0`, etc.).
* `port 5432`: Specify the port number to capture traffic for (in this case, PostgreSQL’s default port).
**Capturing ODBC connection establishment:**
To capture the exact moment when an ODBC connection is established, you can use a filter to capture only the initial handshake packets.
1. Run `tcpdump` with the following options:
„`
tcpdump -n -vv -s 0 -c 1000 -W 1024 -i any port 5432 ‚tcp src port 32773 and tcp dst port 5432‘
„`
Here’s what this filter does:
* `tcp`: Capture TCP packets.
* `src port 32773`: Only capture packets with the source port set to 32773 (a common ODBC port).
* `dst port 5432`: Only capture packets with the destination port set to 5432 (the PostgreSQL port).
**Analyzing captured packets:**
Once you’ve captured the ODBC connection packets, you can use tools like `tcpdump` or `Wireshark` to analyze them. You can filter the captured packets by
protocol (e.g., TCP), port numbers, or specific packet contents.
In Wireshark, for example, you can:
1. Open the capture file.
2. Apply a filter using the „Apply a display filter“ option.
3. Use the `tcp` protocol filter to see only TCP packets.
4. Look for packets with the source port set to 32773 (ODBC) and destination port set to 5432 (PostgreSQL).
By analyzing these packets, you should be able to see the ODBC connection establishment process, including the initial handshake, authentication, and data
transmission.
Keep in mind that capturing network traffic can reveal sensitive information about your system and database. Make sure to use `tcpdump` or other capture
tools responsibly and with caution.
>>> Send a message (/? for help)
>>> /bye
root@pve-ai-llm-01:~#
Proxmox Virtual Environment (VE) 8.2.4 – how to setting up your first local ‚Meta Llama 3‘ Large Language Model (LLM) project without the need for a GPU
Dienstag, Juni 25th, 2024root@pve-ai-llm-01:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 24.04 LTS
Release: 24.04
Codename: noble
root@pve-ai-llm-01:~#
## Installation OLLAMA – a streamlined tool for running open source LLMs locally ##
