|
|
<!DOCTYPE html> |
|
|
<html lang="en"> |
|
|
<head> |
|
|
<meta charset="UTF-8"> |
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0"> |
|
|
<title>UWSN Routing Environment Explained</title> |
|
|
<script src="https://cdn.tailwindcss.com"></script> |
|
|
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.4.0/css/all.min.css"> |
|
|
<style> |
|
|
.gradient-bg { |
|
|
background: linear-gradient(135deg, #1e3a8a 0%, #0ea5e9 100%); |
|
|
} |
|
|
.node-card:hover { |
|
|
transform: translateY(-5px); |
|
|
box-shadow: 0 20px 25px -5px rgba(0, 0, 0, 0.1), 0 10px 10px -5px rgba(0, 0, 0, 0.04); |
|
|
} |
|
|
.code-block { |
|
|
font-family: 'Courier New', monospace; |
|
|
background-color: #1e293b; |
|
|
color: #f8fafc; |
|
|
border-radius: 0.5rem; |
|
|
overflow-x: auto; |
|
|
} |
|
|
</style> |
|
|
</head> |
|
|
<body class="bg-gray-50 min-h-screen"> |
|
|
|
|
|
<header class="gradient-bg text-white py-12"> |
|
|
<div class="container mx-auto px-4"> |
|
|
<div class="flex flex-col items-center"> |
|
|
<i class="fas fa-water text-6xl mb-4"></i> |
|
|
<h1 class="text-4xl md:text-5xl font-bold text-center mb-4">Underwater Wireless Sensor Network Routing</h1> |
|
|
<p class="text-xl text-center max-w-3xl opacity-90"> |
|
|
A Reinforcement Learning Environment for Optimal Packet Routing in UWSNs |
|
|
</p> |
|
|
</div> |
|
|
</div> |
|
|
</header> |
|
|
|
|
|
|
|
|
<main class="container mx-auto px-4 py-12"> |
|
|
|
|
|
<section class="mb-16"> |
|
|
<h2 class="text-3xl font-bold mb-6 text-gray-800 border-b pb-2">Environment Overview</h2> |
|
|
<div class="grid md:grid-cols-2 gap-8"> |
|
|
<div class="bg-white p-6 rounded-lg shadow-md"> |
|
|
<h3 class="text-xl font-semibold mb-4 text-blue-600"> |
|
|
<i class="fas fa-network-wired mr-2"></i> Network Structure |
|
|
</h3> |
|
|
<p class="text-gray-700 mb-4"> |
|
|
The environment models an Underwater Wireless Sensor Network (UWSN) with: |
|
|
</p> |
|
|
<ul class="list-disc pl-6 space-y-2 text-gray-700"> |
|
|
<li><span class="font-medium">N sensor nodes</span> randomly positioned in 3D space</li> |
|
|
<li>One <span class="font-medium">sink node (destination)</span> at a fixed position</li> |
|
|
<li>Each node has <span class="font-medium">limited energy</span> that depletes with transmissions</li> |
|
|
<li>Nodes can only communicate within a <span class="font-medium">transmission range</span></li> |
|
|
</ul> |
|
|
</div> |
|
|
<div class="bg-white p-6 rounded-lg shadow-md"> |
|
|
<h3 class="text-xl font-semibold mb-4 text-blue-600"> |
|
|
<i class="fas fa-robot mr-2"></i> Reinforcement Learning Setup |
|
|
</h3> |
|
|
<p class="text-gray-700 mb-4"> |
|
|
The environment follows the standard Gymnasium interface for RL: |
|
|
</p> |
|
|
<ul class="list-disc pl-6 space-y-2 text-gray-700"> |
|
|
<li><span class="font-medium">State space:</span> Local node info + neighbor metrics</li> |
|
|
<li><span class="font-medium">Action space:</span> Forward to neighbor or drop packet</li> |
|
|
<li><span class="font-medium">Reward function:</span> Balances success with energy, distance, and link quality</li> |
|
|
<li><span class="font-medium">Termination:</span> When packet reaches sink or is dropped</li> |
|
|
</ul> |
|
|
</div> |
|
|
</div> |
|
|
</section> |
|
|
|
|
|
|
|
|
<section class="mb-16"> |
|
|
<h2 class="text-3xl font-bold mb-6 text-gray-800 border-b pb-2">Network Visualization</h2> |
|
|
<div class="bg-white p-6 rounded-lg shadow-md"> |
|
|
<div class="relative h-96 w-full bg-gray-100 rounded-lg overflow-hidden"> |
|
|
|
|
|
<div class="absolute inset-0 flex items-center justify-center"> |
|
|
<div class="text-center"> |
|
|
<i class="fas fa-water text-6xl text-blue-300 mb-4"></i> |
|
|
<p class="text-gray-600">Interactive 3D network visualization would appear here</p> |
|
|
</div> |
|
|
</div> |
|
|
|
|
|
|
|
|
<div class="absolute top-1/4 left-1/4 w-8 h-8 bg-blue-500 rounded-full flex items-center justify-center text-white font-bold shadow-lg"> |
|
|
S |
|
|
</div> |
|
|
<div class="absolute top-1/3 left-1/3 w-6 h-6 bg-green-500 rounded-full flex items-center justify-center text-white font-bold shadow-md"> |
|
|
1 |
|
|
</div> |
|
|
<div class="absolute top-1/2 left-1/2 w-6 h-6 bg-green-500 rounded-full flex items-center justify-center text-white font-bold shadow-md"> |
|
|
2 |
|
|
</div> |
|
|
<div class="absolute top-2/3 left-2/3 w-6 h-6 bg-green-500 rounded-full flex items-center justify-center text-white font-bold shadow-md"> |
|
|
3 |
|
|
</div> |
|
|
<div class="absolute top-3/4 left-1/4 w-6 h-6 bg-green-500 rounded-full flex items-center justify-center text-white font-bold shadow-md"> |
|
|
4 |
|
|
</div> |
|
|
|
|
|
|
|
|
<div class="absolute top-1/4 left-1/4 w-px h-16 bg-blue-300 transform rotate-45 origin-top"></div> |
|
|
<div class="absolute top-1/4 left-1/4 w-px h-24 bg-blue-300 transform -rotate-30 origin-top"></div> |
|
|
<div class="absolute top-1/3 left-1/3 w-px h-16 bg-green-300 transform rotate-15 origin-top"></div> |
|
|
</div> |
|
|
|
|
|
<div class="mt-6 grid grid-cols-2 md:grid-cols-4 gap-4"> |
|
|
<div class="bg-blue-50 p-3 rounded-lg flex items-center"> |
|
|
<div class="w-3 h-3 bg-blue-500 rounded-full mr-2"></div> |
|
|
<span>Sink Node</span> |
|
|
</div> |
|
|
<div class="bg-green-50 p-3 rounded-lg flex items-center"> |
|
|
<div class="w-3 h-3 bg-green-500 rounded-full mr-2"></div> |
|
|
<span>Sensor Nodes</span> |
|
|
</div> |
|
|
<div class="bg-blue-100 p-3 rounded-lg flex items-center"> |
|
|
<div class="w-3 h-3 bg-blue-300 rounded-full mr-2"></div> |
|
|
<span>Active Links</span> |
|
|
</div> |
|
|
<div class="bg-gray-100 p-3 rounded-lg flex items-center"> |
|
|
<div class="w-3 h-3 bg-gray-400 rounded-full mr-2"></div> |
|
|
<span>Inactive Links</span> |
|
|
</div> |
|
|
</div> |
|
|
</div> |
|
|
</section> |
|
|
|
|
|
|
|
|
<section class="mb-16"> |
|
|
<h2 class="text-3xl font-bold mb-6 text-gray-800 border-b pb-2">Key Environment Components</h2> |
|
|
|
|
|
|
|
|
<div class="bg-white p-6 rounded-lg shadow-md mb-8"> |
|
|
<h3 class="text-2xl font-semibold mb-4 text-blue-600"> |
|
|
<i class="fas fa-database mr-2"></i> State Space |
|
|
</h3> |
|
|
<p class="text-gray-700 mb-4"> |
|
|
The observation space provides the agent with information about: |
|
|
</p> |
|
|
|
|
|
<div class="grid md:grid-cols-2 gap-6"> |
|
|
<div> |
|
|
<h4 class="font-medium text-lg mb-2 text-gray-800">Local Node Information</h4> |
|
|
<div class="code-block p-4 mb-4"> |
|
|
<pre>{ |
|
|
"energy": 50.0, # Current node's remaining energy |
|
|
"position": [x, y, z] # 3D coordinates of current node |
|
|
}</pre> |
|
|
</div> |
|
|
</div> |
|
|
<div> |
|
|
<h4 class="font-medium text-lg mb-2 text-gray-800">Neighbor Information</h4> |
|
|
<div class="code-block p-4"> |
|
|
<pre>{ |
|
|
"energy": 45.0, # Neighbor's remaining energy |
|
|
"distance": 25.3, # Distance to neighbor |
|
|
"RSSI": -75.2, # Received Signal Strength |
|
|
"SNR": 15.8, # Signal-to-Noise Ratio |
|
|
"PDR": 0.92 # Packet Delivery Ratio |
|
|
}</pre> |
|
|
</div> |
|
|
</div> |
|
|
</div> |
|
|
</div> |
|
|
|
|
|
|
|
|
<div class="bg-white p-6 rounded-lg shadow-md mb-8"> |
|
|
<h3 class="text-2xl font-semibold mb-4 text-blue-600"> |
|
|
<i class="fas fa-running mr-2"></i> Action Space |
|
|
</h3> |
|
|
<p class="text-gray-700 mb-4"> |
|
|
The agent can choose from the following actions: |
|
|
</p> |
|
|
|
|
|
<div class="grid md:grid-cols-3 gap-4"> |
|
|
<div class="node-card bg-green-50 p-4 rounded-lg border border-green-200 transition-all duration-300"> |
|
|
<div class="flex items-center mb-2"> |
|
|
<div class="w-8 h-8 bg-green-500 rounded-full flex items-center justify-center text-white font-bold mr-3">N</div> |
|
|
<h4 class="font-medium">Forward to Neighbor N</h4> |
|
|
</div> |
|
|
<p class="text-sm text-gray-600">Transmit packet to selected neighbor node</p> |
|
|
</div> |
|
|
<div class="node-card bg-blue-50 p-4 rounded-lg border border-blue-200 transition-all duration-300"> |
|
|
<div class="flex items-center mb-2"> |
|
|
<div class="w-8 h-8 bg-blue-500 rounded-full flex items-center justify-center text-white font-bold mr-3">0</div> |
|
|
<h4 class="font-medium">Forward to Sink</h4> |
|
|
</div> |
|
|
<p class="text-sm text-gray-600">Direct transmission to destination node</p> |
|
|
</div> |
|
|
<div class="node-card bg-red-50 p-4 rounded-lg border border-red-200 transition-all duration-300"> |
|
|
<div class="flex items-center mb-2"> |
|
|
<div class="w-8 h-8 bg-red-500 rounded-full flex items-center justify-center text-white font-bold mr-3">X</div> |
|
|
<h4 class="font-medium">Drop Packet</h4> |
|
|
</div> |
|
|
<p class="text-sm text-gray-600">Terminate transmission (penalized)</p> |
|
|
</div> |
|
|
</div> |
|
|
</div> |
|
|
|
|
|
|
|
|
<div class="bg-white p-6 rounded-lg shadow-md"> |
|
|
<h3 class="text-2xl font-semibold mb-4 text-blue-600"> |
|
|
<i class="fas fa-coins mr-2"></i> Reward Function |
|
|
</h3> |
|
|
<p class="text-gray-700 mb-4"> |
|
|
The reward balances multiple objectives to encourage efficient routing: |
|
|
</p> |
|
|
|
|
|
<div class="code-block p-4 mb-6"> |
|
|
<pre>reward = R_success # Base reward for success |
|
|
- β * distance² # Penalize long distances |
|
|
- η * energy_consumed # Penalize high energy use |
|
|
- δ * (1 - PDR) # Penalize poor link quality |
|
|
- θ * (1 / residual_energy) # Penalize low-energy nodes</pre> |
|
|
</div> |
|
|
|
|
|
<div class="grid md:grid-cols-2 gap-6"> |
|
|
<div> |
|
|
<h4 class="font-medium text-lg mb-2 text-gray-800">Reward Components</h4> |
|
|
<ul class="space-y-3"> |
|
|
<li class="flex items-start"> |
|
|
<div class="bg-blue-100 p-1 rounded-full mr-3 mt-1"> |
|
|
<i class="fas fa-check-circle text-blue-500 text-sm"></i> |
|
|
</div> |
|
|
<span><span class="font-medium">R_success:</span> Fixed reward for reaching sink</span> |
|
|
</li> |
|
|
<li class="flex items-start"> |
|
|
<div class="bg-red-100 p-1 rounded-full mr-3 mt-1"> |
|
|
<i class="fas fa-times-circle text-red-500 text-sm"></i> |
|
|
</div> |
|
|
<span><span class="font-medium">Distance penalty:</span> Discourages long hops</span> |
|
|
</li> |
|
|
<li class="flex items-start"> |
|
|
<div class="bg-red-100 p-1 rounded-full mr-3 mt-1"> |
|
|
<i class="fas fa-times-circle text-red-500 text-sm"></i> |
|
|
</div> |
|
|
<span><span class="font-medium">Energy penalty:</span> Conserves network energy</span> |
|
|
</li> |
|
|
<li class="flex items-start"> |
|
|
<div class="bg-red-100 p-1 rounded-full mr-3 mt-1"> |
|
|
<i class="fas fa-times-circle text-red-500 text-sm"></i> |
|
|
</div> |
|
|
<span><span class="font-medium">Link quality penalty:</span> Encourages reliable paths</span> |
|
|
</li> |
|
|
</ul> |
|
|
</div> |
|
|
<div> |
|
|
<h4 class="font-medium text-lg mb-2 text-gray-800">Example Scenarios</h4> |
|
|
<ul class="space-y-3"> |
|
|
<li class="flex items-start"> |
|
|
<div class="bg-green-100 p-1 rounded-full mr-3 mt-1"> |
|
|
<i class="fas fa-arrow-up text-green-500 text-sm"></i> |
|
|
</div> |
|
|
<span><span class="font-medium">Short, efficient path:</span> High reward</span> |
|
|
</li> |
|
|
<li class="flex items-start"> |
|
|
<div class="bg-yellow-100 p-1 rounded-full mr-3 mt-1"> |
|
|
<i class="fas fa-equals text-yellow-500 text-sm"></i> |
|
|
</div> |
|
|
<span><span class="font-medium">Long but reliable path:</span> Moderate reward</span> |
|
|
</li> |
|
|
<li class="flex items-start"> |
|
|
<div class="bg-red-100 p-1 rounded-full mr-3 mt-1"> |
|
|
<i class="fas fa-arrow-down text-red-500 text-sm"></i> |
|
|
</div> |
|
|
<span><span class="font-medium">Dropped packet:</span> Significant penalty</span> |
|
|
</li> |
|
|
</ul> |
|
|
</div> |
|
|
</div> |
|
|
</div> |
|
|
</section> |
|
|
|
|
|
|
|
|
<section> |
|
|
<h2 class="text-3xl font-bold mb-6 text-gray-800 border-b pb-2">Technical Implementation</h2> |
|
|
|
|
|
<div class="grid md:grid-cols-2 gap-8"> |
|
|
|
|
|
<div class="bg-white p-6 rounded-lg shadow-md"> |
|
|
<h3 class="text-xl font-semibold mb-4 text-blue-600"> |
|
|
<i class="fas fa-cogs mr-2"></i> Environment Initialization |
|
|
</h3> |
|
|
<div class="code-block p-4 mb-4"> |
|
|
<pre>def __init__(self, num_nodes=10, max_energy=100, |
|
|
initial_energy=50, transmission_range=100): |
|
|
# Initialize node positions randomly in 3D space |
|
|
self.node_positions = np.random.uniform(0, 100, size=(num_nodes, 3)) |
|
|
|
|
|
# Set initial energies |
|
|
self.node_energies = np.full(num_nodes, initial_energy) |
|
|
|
|
|
# Define action space (forward to any node or drop) |
|
|
self.action_space = gym.spaces.Discrete(num_nodes + 1) |
|
|
|
|
|
# Define observation space (local + neighbor info) |
|
|
self.observation_space = gym.spaces.Dict(...)</pre> |
|
|
</div> |
|
|
<p class="text-gray-700"> |
|
|
The environment is initialized with configurable parameters for network size, energy levels, and transmission range. |
|
|
</p> |
|
|
</div> |
|
|
|
|
|
|
|
|
<div class="bg-white p-6 rounded-lg shadow-md"> |
|
|
<h3 class="text-xl font-semibold mb-4 text-blue-600"> |
|
|
<i class="fas fa-code-branch mr-2"></i> Step Function |
|
|
</h3> |
|
|
<div class="code-block p-4 mb-4"> |
|
|
<pre>def step(self, action): |
|
|
if action == "drop": |
|
|
reward = -10 |
|
|
terminated = True |
|
|
elif action == sink_node: |
|
|
reward = R_success |
|
|
terminated = True |
|
|
else: |
|
|
# Calculate multi-component reward |
|
|
reward = compute_reward(current_node, action) |
|
|
# Update node energy |
|
|
self.node_energies[action] -= 1 |
|
|
|
|
|
return observation, reward, terminated, truncated, info</pre> |
|
|
</div> |
|
|
<p class="text-gray-700"> |
|
|
The step function handles packet forwarding, energy consumption, and computes the multi-objective reward. |
|
|
</p> |
|
|
</div> |
|
|
</div> |
|
|
</section> |
|
|
</main> |
|
|
|
|
|
|
|
|
<footer class="gradient-bg text-white py-8"> |
|
|
<div class="container mx-auto px-4"> |
|
|
<div class="flex flex-col md:flex-row justify-between items-center"> |
|
|
<div class="mb-4 md:mb-0"> |
|
|
<h3 class="text-xl font-bold">UWSN Routing Environment</h3> |
|
|
<p class="text-blue-100">A Gymnasium-compatible RL environment</p> |
|
|
</div> |
|
|
<div class="flex space-x-4"> |
|
|
<a href="#" class="text-white hover:text-blue-200 transition"> |
|
|
<i class="fab fa-github text-2xl"></i> |
|
|
</a> |
|
|
<a href="#" class="text-white hover:text-blue-200 transition"> |
|
|
<i class="fab fa-python text-2xl"></i> |
|
|
</a> |
|
|
<a href="#" class="text-white hover:text-blue-200 transition"> |
|
|
<i class="fas fa-book text-2xl"></i> |
|
|
</a> |
|
|
</div> |
|
|
</div> |
|
|
<div class="mt-6 text-center text-blue-100 text-sm"> |
|
|
<p>© 2023 Underwater Wireless Sensor Network RL Project</p> |
|
|
</div> |
|
|
</div> |
|
|
</footer> |
|
|
|
|
|
<script> |
|
|
|
|
|
document.querySelectorAll('.node-card').forEach(card => { |
|
|
card.addEventListener('mouseenter', () => { |
|
|
card.classList.add('shadow-lg'); |
|
|
}); |
|
|
card.addEventListener('mouseleave', () => { |
|
|
card.classList.remove('shadow-lg'); |
|
|
}); |
|
|
}); |
|
|
</script> |
|
|
<p style="border-radius: 8px; text-align: center; font-size: 12px; color: #fff; margin-top: 16px;position: fixed; left: 8px; bottom: 8px; z-index: 10; background: rgba(0, 0, 0, 0.8); padding: 4px 8px;">Made with <img src="https://enzostvs-deepsite.hf.space/logo.svg" alt="DeepSite Logo" style="width: 16px; height: 16px; vertical-align: middle;display:inline-block;margin-right:3px;filter:brightness(0) invert(1);"><a href="https://enzostvs-deepsite.hf.space" style="color: #fff;text-decoration: underline;" target="_blank" >DeepSite</a> - 🧬 <a href="https://enzostvs-deepsite.hf.space?remix=Ihabaa/mdp" style="color: #fff;text-decoration: underline;" target="_blank" >Remix</a></p></body> |
|
|
</html> |
